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ABSTRACT 


^-Thm  purpoM  of  this  thesis  is  to  present  and  discuss  a 
ssthod  of  assessing  ths  affactivanasa  of  a  work  spaca 
layout.  In  addition,  this  sathod  will  provida  tha  framework 
for  pinpointing  thosa  araas  of  layout  dasign  whara  radasign 
will  ba  most  cost  affective.  The  objective  is  to  address 
inefficiencies  in  the  layout  of  warfare  modules  on  U.S.  Navy 
combatants.  In  particular,  tha  Electronic  Warfare  Module  on 
aircraft  carriers  is  assessed  due  to  tha  highly  time- 
critical  nature  of  electronic  warfare.  Tha  method  chosen  in 
this  thesis  is  a  modification  of  two  techniques  of 
assessment:  Integration  Analysis  and  Mission  Operability 
Assessment  Technique  <MOAT> .  Tha  portions  of  these 
techniques  used  are  Link  Analysis,  Task  Analysis,  and 
Operability  Analysis.  The  application  herein  concludes  that 
the  EW  Module  layout  design  on  the  latest  NIMITZ-class 
aircraft  carriers  was  less  than  40*  effective  in  promoting 


mission  accomplishment. 
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A.  BACKGROUND 

As  smd  in  several  of  the  rcctnt  wars  and  conflicts , 
spssd  and  timing  ars  crucial  in  nodsrn  warfare.  In  the 
Falklands  War,  the  lack  of  tine  available  to  react  to  a 
threat  caused  the  loss  of  HNS  SHEFFIELD.  The  HNS  SHEFFIELD 
was  sunk  by  fires  that  could  not  be  brought  under  control  as 
a  result  of  a  strike  by  an  Exocet  aissile.  Even  though  the 
ship  had  weapons  systeaa  that  could  have  defeated  the 


Exocet,  its  inability  to  initially  detect  the  missile  at  a 
far  range  rendered  these  defenses  useless.  The  Electronic 
Warfare  <EW)  operators  on  the  SHEFFIELD  had  little  warning 
of  the  Exocet  due  to  self-induced  jamming.  When  the  self- 


jaaaing  (inadvertent,  of  course)  ceased,  the  Exocet  was 
iaaediately  detected,  but  it  was  too  late  to  engage.  The 
missile  struck  about  ten  seconds  later.  Although 
technologically  superior,  the  British  did  not  correctly 
manage  the  Radio  Frequency  (RF)  spectrum  and  lost  a  ship. 
The  self-jamming  was  caused  by  equipment  interference  and 
was  either  not  noticed  earlier  or  dismissed  as  unlikely  to 
cause  serious  problems.  This  problem  and  others  like  it  are 


now  being  resolved  by  the  British. 

In  war  at  sea  today,  it  is  necessary  to  provide  an 
adequate  reaction  time.  Reaction  time  is  the  time  between 


detection  of  the  incoming  target  and  weapona  angagaaant. 
Raaponaa  tiaa  ia  tha  time  between  eneay  weapona  releaae  and 
impact  (i.e.,  the  time  available  for  detection,  reaction, 
and  engagement).  In  the  example  cited  above,  that  adequate 
reaction  time  aiaply  waa  not  preaent.  In  the  case  of  HMS 
SHEFFIELD,  improper  management  of  the  electromagnetic  <EM) 
apectrum  aet  up  the  aituation  of  inadequate  reaction  time. 
With  the  coming  of  aophiaticated  weapon  systems  and 
supersonic  miaailea,  the  amount  of  time  available  to  respond 
to  a  threat  has  been  steadily  reduced.  In  World  War  II 
reaction  time  could  be  measured  on  the  order  of  dozens  of 
minutes.  Today  reaction  times  are  on  the  order  of  dozens  of 
seconds.  With  initial  detection  at  the  horizon,  sea- 
skimming  missiles  offer  only  30  seconds  warning  before 
impact.  Today's  Combat  Information  Center  <CIC)  needs  to  be 
organized  in  such  a  way  as  to  derive  maximum  efficiency  and 
speed  from  operations  in  order  to  reduce  reaction  time  as 
much  as  possible. 

The  problem  of  reduced  reaction  time  is  not  new  and 
equipments  in  many  areas  has  been  developed  to  meet  this 
need.  Tha  Navy  Tactical  Data  System  (NTDS)  was  developed  to 
solve  this  problem.  It  reduces  the  amount  of  time  needed  to 
understand  the  tactical  environment  around  the  ship  and  by 
providing  a  more  complete  picture,  aids  sound  decision 


making 


With  the  development  and  installation  of  affactiva 
dafanaiva  weapon  ayataaa  onboard  Navy  ships,  affort  aust  now 
ba  davotad  to  tha  raduction  of  raaction  tiaa.  Eff active 
waapona  ara  available,  only  tha  tiaa  to  aaploy  thaa 
corractly  ia  naadad.  Effactiva  long  ranga  sanaor  systans 
can  provida  adequate  warning  and  MbuyM  tiaa  for  the 
aaployaant  of  tha  appropriate  waapon.  Therefore,  it  can  be 
concluded  that  anything  that  “buys  tiaa**  is  of  value. 

But  how  does  one  buy  tiaa?  There  ara  two  ways:  (1) 
machines  can  ba  built  to  react  more  quickly  or,  (2) 
operators  can  ba  trained  to  respond  faster.  Although 
systaas  will  help  tha  Fleet  sailor  react  quicker,  there  is  a 
liait  as  to  how  fast  ha  can  respond.  Working  spaces  need  to 
ba  optimized  so  that  tha  sailor  can  respond  optimally.  In 
this  context,  efficiency  translates  into  speed  which 
translates  into  reduced  raaction  tiaa.  Tha  efficient 
arrangement  of  aquipaant  in  a  working  space  has  not  been 
addressed  by  tha  Navy  in  such  a  way  as  to  promote  affective 
and  efficient  spaces.  (For  tha  remainder  of  this  affort, 
tha  term  “working  space'*  or  “work  space*'  will  be  used  to 
denote  a  combat  space  where  data  is  searched  for,  collected, 
evaluated,  disseminated,  and/or  acted  upon.)  An  efficient 
and  effectively  laid  out  workspace  will,  intuitively,  buy 
time.  The  barriers  imposed  by  improper  design  and  poor 
layout  can  never  be  totally  compensated  for  by  training. 
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During  the  procurMtnt  cycle,  there  Is  a  requirement  to 
perform  human  factors  snglnssrlng  on  all  new  squlpssnt  to 
insure  an  adequate  aan-aachine  interface.  However,  there  is 
Inadequate  methodology  for  lnauring  that  space  arrangement 
contributes  to  successful  aission  accomplishment.  Space 
arrangement  is,  apparently,  dictated  primarily  by  the  need 
to  fit  new  and  existing  equipment  into  a  space.  This  is  not 
intended  to  belittle  the  efforts  of  those  who  are  charged 
with  designing  the  layouts  and  arrangements  of  combat 
spaces,  but  is  intended  to  address  methods  for  improving  the 
efficiencies  of  layouts. 

Before  the  layouts  of  combat  spaces  can  be  redesigned, 
it  is  first  necessary  to  determine  if  there  is  a  deficiency 
in  the  existing  layout.  A  measure  of  adequacy  of  space 
arrangements  must  be  developed.  At  present  there  is  no  such 
measure  for  combat  systems  layouts. 

The  field  of  huaan  factors  engineering  has  developed 
techniques  for  assessing  the  adequacy  of  tasks,  subsystems, 
systems,  and  organizations.  However,  due  to  the  dynamic 
nature  of  shipboard  work  space  development  there  appears  to 
have  been  few  human  factors  engineering  techniques 
addressing  the  arrangement  of  systems  as  applied  to  the 
space  as  a  whole  or  the  entire  mission  work  areas.  This  is 
due  to  the  fact  that  new  equipment  with  new  functions  and 


incrMMd  capabilities  «rt  constantly  being  introduced  into 
spaces  barely  adequate  for  the  original  aquipsent. 

As  a  result  of  the  procurement  cycle,  human  factors 
engineering  is  applied  only  on  single  systems  or  consoles. 
It  has  bean  recognized  that  training  personnel  to  overcome 
the  human  factors  design  deficiencies  is  not  cost  effective 
in  terns  of  either  time,  money,  or  manpower.  The  current 
requirement  is  for  total  individual  system  analysis  in  the 
areas  of  compatibility,  interoperability,  and  human  factors. 
It  is  mainly  in  the  area  of  integrating  these  systems 
together  in  a  work  space  that  significant  improvement  is 
needed. 

C.  PURPOSE  OF  THESIS 

The  purpose  of  this  thesis  is  to  present  and  discuss  a 
method  of  assessing  the  effectiveness  of  a  work  space 
layout.  In  addition,  this  method  will  provide  the  framework 
for  pinpointing  those  areas  where  redesign  will  be  most  cost 
effective. 


proposed.  That.  solution  apparently  waa  forwarded  to 
NAVSBA3YSC0N  and  Spaea  Naval  Warfare  Coaaand  and  than  to 
QPNAV  because  it  has  bean  incorporated  in  the  layout  of  the 
EW  Nodule  on  CVN-71  (Sea  Figure  1). 

The  other  aathod  in  improving  apace  design  ia  a  aock-up 
approach.  At  various  organizations,  mock-ups  are  used  to 
teat  the  layout  designs  considered.  These  organizations  are 
under  contract  to  produce  a  specific  kind  of  aock-up.  The 
Naval  Ocean  Systaaa  Canter  (NOSC)  in  San  Diego  does  some 
aock-up  work  under  the  direction  of  NAVSEASYSCOM .  At 
present  they  have  a  carrier  Coabat  Information  Center  (CIC) 
aock-up.  It  contains  the  Display  and  Decision  portion  which 
includes  the  Surface  Warfare  Nodule  and  the  Air  Warfare 
Nodule.  It  does  not  contain  Detection  and  Tracking.  the 
Electronic  Warfare  Nodule,  or  the  Anti-Subaarine  Warfare 
Nodule. 

B.  DEFICIENCIES 

The  currant  aathod  of  improving  the  layout/arrangeaent 
design  of  some  of  the  spaces  on  our  surface  ships.  has  four 
major  shortcomings:  (1)  lack  of  user  input.  (2)  lack  of 
human  factors  engineering.  (3)  lack  of  a  learning  curve,  and 
(4)  no  planning  for  growth.  These  deficiencies  and  the 
impact  they  have  on  the  effectiveness  of  the  space  is  now 
discussed. 
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The  currant  method  of  layout  improvement  haa  little 


or  no  operator /user  input.  There  ia.  perhaps,  some  uaer 
input  aa  in  the  CARL  VINSON  CIC  example  cited  above  but  this 
appears  to  be  the  exception  rather  than  the  rule.  There  ia 
no  formal  method  of  submitting  a  design  change  through 
normal  Navy  channels  to  NAVSEA.  This  ia  a  very  serious 
deficiency  because  sketchy  or  total  lack  of  fleet  input  ia 
counter-productive.  The  design  of  a  space  by  those  who  do 
not  and  will  not  be  using  it  haa  a  tendency  to  result  in  a 
far  from  optimal  design.  For  example.  a  radio  or  “bitch 
box**  that  ia  frequently  used  is  placed  just  out  of  reach. 
In  the  EW  Module  on  USS  CARL  VINSON.  the  12  MC  (internal 
communications  set)  used  to  communicate  with  the  Tactical 
Action  Officer  (TAO)  and  INTEL(aaong  others)  is  placed  such 
that  the  EW  Supervisor  and  NTOS  operator  have  to  get  up  out 
of  hia  seat  to  talk.  There  is.  in  fairness.  a  hand  mike 
that  can  be  attached  but  this  haa  the  undesirable  side 
effect  of  cluttering  the  workstation. 

2.  Lack  of  Human  Factors  Engineering 

The  only  human  factors  engineering  being  employed  is 
basic.  This  method  has  been  characterized  as  “moving  the 
furniture  around”.  This  is  done  until  there  is  an 
apparently  workable  solution.  Again  there  is  inadequate 
fleet  or  operator  inputs  to  check  the  “new**  arrangement.  It 
must  be  aald  that  thoae  employing  this  method  are  trying  to 


find  an  arrufMmt  that  facilitates  an  efficient  and 
effective  oparation.  Thia  aathod  ta  tha  baat  aathod 
eurrantly  aval labia  to  aceoapliah  thia  task.  but  atill 
something  ia  lacking. 

3.  Lack  of  a  Looming  Curva 

However  good  tha  raaulta  of  tha  aock-up  aathod  aay 
ba.  thara  ia  no  apparent  learning  curva  in  auccaaaiva 
layouta.  For  example.  tha  EW  Hodula  layout  on  USS  AMERICA 
provided  the  nocoaaary  room  for  the  activities  of  EW  and 
gava  tha  iapraaaion  of  aaooth  efficiency  and  coapetence. 
However,  on  later  aircraft  carriers  (moat  notably,  the 
NIMITZ  class)  the  EW  Module  arrangement  is  a  regression  and 
in  nowise  approaches  the  room  and  layout  effectiveness  found 
on  AMERICA.  If  the  NIMITZ  class  carrier  EW  Nodule  layout 
was  intended  to  be  an  improvement  over  AMERICA,  it  failed. 

4.  Wo  elm  for  Qrgvth 

The  current  method  is  deficient  in  terms  of  its 
potential  for  growth.  Few  designs  provide  room  for 
expansion  for  either  new  equipment  or  modifications  to  older 
equipment.  When  new  equipment  is  added.  the  space  for  it 
must  come  from  someplace.  even  if  that  area  has  another 
function.  Simply  adding  new  equipment  does  not  aid  the 
operation  of  the  overall  work  area  and  may  even  be 
counterproductive  in  that  efficiency  nay  be  reduced. 

When  lead  units  are  designed  and  built,  they  are 


constructed  with  only  existing  equipment  is  mind  (this  is  a 


gmtral  rule  and  than  arc  sou  exceptions) .  There  is  aoaa 
sssll  amount,  of  room  for  axpanalon  of  capabilitiaa  but  it  ia 
thought  that  tha  now  equipment  will  raplaca  oldar  aquipaant 
and  taka  up  tha  aaaa  amount  (or  lama)  of  room  and  fit  into 
tha  aaaa  apaea.  This  thought  doaa  not  taka  into  account  naw 
miaaiona  for  tha  apaca  with  corresponding  naw  aquipaant,  naw 
capabilitiaa,  and  naw  apaca  raquiramanta.  Tharafora,  ona 
can  raadlly  aaa  that  naw  aquipaant  must  ba  addad  wharavar 
thara  ia  room  for  it.  Soaatiaaa  tha  apaca  whara  tha  naw 
aquipaant  ia  addad  ia  unsatisfactory  for  tha  aquipaant  and 
ita  oparation.  By  way  of  axaapla,  USS  CARL  VINSON  ia  a 
NIMITZ  claaa  aircraft  carriar  and  waa  built  using 
eaaantially  tha  aaaa  bluaprinta  aa  tha  laad  ahip.  Tha  EW 
Module  apaca  waa  not  changed  avan  though  tha  SLQ-17  and  SSQ- 
82  (MUTE) ,  not  yat  procured  whan  NIMITZ  waa  daaignad  and 
built,  ware  alatad  for  CARL  VINSON  (and  all  carriara, 
eventually) (Saa  Figure  2) .  Only  ona  aquipaant  rack  waa 
removed  to  aaka  way  for  both  equipmenta.  Tha  SLQ-17  conaola 
fita  into  tha  vacated  equipment  rack.  Thia  atill  left  tha 
computer  unit  (about  ona  and  a  half  racka)  and  MUTE  (which 
waa  daaignad  and  built  wider  than  tha  atandard  rack)  to  ba 
placed  aomawhara  within  the  EW  Module.  Tha  SLQ-17  computer 
rack  waa  placed  against  tha  bulkhead  in  tha  middle  of  the 
apaca.  At  moat,  three  operators  could  fit  comfortably  into 
tha  EW  Module  avan  though  General  Quarters  manning  calls  for 
four  operators,  and  MUTE  waa  placed  outside  the  EW  Module  in 
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Figure  2,  USS  CARL  VINSON  EW  Modul 


a  passageway  within  CIC  that  was  haavily  travalad.  Thara 
was  siaply  no  rooa  in  tha  EW  Hoduia  for  both  tha  EW 
oparatora  and  MUTE — ona  or  tha  othar  had  to  go. 

In  addition  to  tha  problaaa  citad  abova,  to 
accoaaodata  tha  inclusion  of  all  tha  aquipaant  in  tha  EW 
Hoduia*  soaa  aavara  apaca  aconoaiaa  had  to  ba  aada.  Tha 
layout  now  took  on  tha  appaaranca  as  shown  in  Figura  2.  To 
allow  soaa  pasaaga  of  oparatora  and  aaintananca  paopla  aaong 
and  around  tha  aquipaant*  a  “straight  lina'*  layout  was 
adoptad.  This  had  tha  sola  advantaga  of  allowing  all  tha 
aquipaant  possibla  to  ba  placa  in  tha  spaca.  Howavar*  tha 
quastion  can  logically  ba  askad*  "Does  such  an  arrangeaant 
add  or  detract  froa  tha  afficiancy  and  affactivanass  of 
spaca  utilization  in  accoaplishinr;  tha  aission?"  New 
aquipaant  addad  to  a  spaca  that  was  not  dasignad  for  it  aay 
causa  integration  problaaa  due  to  its  intrinsic  nature 
(i.e.*  in  the  aquipaant  itself)*  its  new  location  (e.g.*  tha 
SLQ-17  coaputer  rack),  and  reduced  workspace  (in  our 
exaaple*  several  racks  where  ona  used  to  ba  to  tha  exclusion 
of  another  piece  of  aquipaant- -MUTE) . 

Tha  reaainder  of  this  thesis  will  ba  given  over  to 
atteapting  to  find  a  workable  solution  to  the  problea  of 
adequately  designing  a  work  spaca*  in  particular*  an  EW 
Hoduia.  As  indicated  earlier  this  is  an  area  where  the  costs 
are  in  dollars  and  effort*  but  the  payoff  is  in  shorter 
reaction  tiae  and*  ultiaately*  in  ships  and  lives  saved. 


A.  IMPROVEMENT  TO  LAYOUTS 


The  solution  to  layout /arrangement  improvement  is 
naithar  simple  nor  straightforward.  An  improvement, 
howavar ,  can  ba  found  in  a  thrssfold  approach  to  tha 
problaa.  Thasa  ara:  (1)  a  ship  class,  module,  nock -up  at  a 
land  basad  laboratory,  (2)  flaat  inputs  addad  to  it  on  a 
ragular  basis,  and  (3)  a  quantifiabla  naasura  that  can  be 
used  to  deternine  overall  effectiveness  and  pinpoint  problem 
areas . 

Establishing  a  class,  nodule,  nock-up  at  a  land  basad 
laboratory  nakas  good  sense.  Hare,  tha  results  of  several 
nock-upa  can  be  stored  and  compared.  Here,  too,  a  “learning 
curve'*  can  be  established.  What  does  not  work  for  one  class 
and  nodule  nay  never  work,  or  it  nay  work  for  another  class 
ship  and  another  nodule.  The  cost  of  nock-ups  can  be  kept 
low.  Mock-ups  of  new  equipment  entering  the  fleet  can  be 
sent  to  just  one  location  and  then  incorporated  into  the 
design  or  redesign.  Mock-ups  of  new  ship  classes  can  easily 
be  done  there. 

NOSC  at  San  Diego  seems  to  be  a  good  place  to  have  this 
nock-up  facility  for  several  reaaons.  First,  experts  there 
have  already  done  none  nock -up  work  and  have  a  certain 
amount  of  experience  in  this  area.  Secondly,  they  are  near 


a  good  sourca  of  float  Inputa  In  San  Dlago 


Onca  a  nock-up 


was  daaignad  (or  radaaignad) ,  NOSC  could  raquaat  aoaa  float, 
oparatora  from  ona  of  the  ahlpa  of  that  particular  claaa  and 
thaaa  oparatora  could  critlqua  tha  mock-up  and  mako 
auggaatlona  for  lmprovamant.  For  addad  raalism  and 
additional  inputa,  a  mock  acanario  could  ba  playad  out  by 
tha  oparatora  on  tha  mock-up.  Thia  haa  tha  addad  poaaible 
banafit  of  uncovaring  any  ovaraight  by  aithar  NOSC  or  the 
oparatora'  critlqua.  Tha  two  logical  placaa  for  tha  mock-up 
aita  ara  Norfolk,  Va.  and  San  Diago,  Ca. 

Fleet  inputa  in  tha  daaign/radaaign  of  tha  layout 
procaaa  ia  of  tha  utmoat  importance.  Tha  fleet  operators 
are  the  people  who  have  to  uaa  the  equipment  and  accomplish 
the  mission  within  the  space.  They,  from  the  benefit  of 
several  years  individual  and  many  years  collective 
experience,  will  be  able  to  note  problems  with  the  mock-up 
that  the  designers  may  have  missed.  Designers  of  single 
equipments  tend  to  think  of  their  equipment  in  isolation 
from  all  others.  Layout  designers  are  often  not  familiar 
with  the  operating  characteristics  of  all  the  equipment. 
Fleet  operators  suffer  from  neither  of  these  deficiencies. 
However,  operators  do  have  a  bias  toward  doing  things  as 
they  are  currently  done  and  may  resist  change.  Nevertheless, 
they  are  still  probably  the  beat  ones  to  evaluate  the  mock- 


As  sMRingly  complete  a*  the  combination  of  both 
laboratory  aock-up  and  fleet  input  night  be*  thara  is  ona 
nora  araa  that  naads  to  ba  covarad.  This  is  a  quantitative 
asaassnant  of  tha  praaant  layout  and  nock-up  layouts.  Thara 
ara  savaral  raasons  for  this.  First*  a  quantitativa 
asaassnant  of  a  prasant  layout  nay  indicata  that  it  doas  not 
naad  inproving  or  that  tha  cost  of  inproving  tha  layout  is 
not  justified  by  tha  anount  of  inprovanant.  Second*  a 
quantitativa  asaassnant  based  in  part  on  questionnaires  to 
fleet  operators  nay  awaken  thoughts  of  sons  inadequacy  that 
was  not  prasant  in  tha  conscious  nanory  but  was  tucked  away 
in  tha  racassas  of  tha  nind.  Finally*  a  quantitative 
asaassnant  is  necessary  to  ba  able  to  conpara  functional 
layouts  ona  to  another.  Tha  final  aspect  of  this  approach  is 
a  way  of  assessing  tha  effectiveness  of  tha  layout. 

Various  techniques  have  bean  developed  that  will  aid  in 
assessing  affactivanaas.  However*  these  nathods  have  been 
used  on  systens  that  ara  disainilar  to  those  found  on  ships 
and  nust  ba  nodifiad.  Tha  nathod  that  will  ba  utilized  is  a 
conbination  of  three  different  but  related  techniques:  Task 
Analysis*  Link  Analysis*  and  Operability  Analysis.  Two 
major  studies  have  been  reviewed  to  determine  the  extent  of 
these  analyses  and  how  they  night  ba  nodifiad  for  a  layout 
inprovanant  application.  These  ara  Integration  Analysis  and 
Mission  Operability  Asaassnant  Technique  (MOAT).  A  brief 


look  at  aach  of  thaaa  will  indicate  tha  aallant  portiona  of 


each  for  thla  application. 

Integration  Analyala  la  tha  integration  of  Taak 
Analyala,  Operator  Interviewa,  and  Link  Analyala  to  evaluate 
a  ayatem 'a  Functional  Mock-up.  Integration  Analyala  was 
deaigned  aa  a  viable  Teat  and  Evaluation  technique  for  the 
earlier  atagea  of  Oevelopaantal  Teat  and  Evaluation  (DT&E) 
in  order  to  reduce  dealgn  diacrepanciea  and  minimize 
acquiation  coata  and  time  CRaf .  13. 

MOAT,  an  evaluation  methodology,  meaaurea  the 
operability  of  a  ay atom  or  aubayatem  in  terms  of  operator 
taaka  performed  during  a  aiaaion.  It  eaeentially  ia  an 
Operability  Analyala. 

In  general,  MOAT  addreaaea  the  problem  of  how  well  an 
operator  can  uae  a  ayatem  or  aubayatem  to  perform  taaka 
within  the  miaaion  context.  Contraated  to  evaluationa 
uaing  human  engineering  dealgn  criteria  which  preaent  only 
paaa  or  fail  information,  thla  technique  providea 
information  on  the  degree  of  ayatem  and/or  aubayatem 
aucceaa  or  failure. CRef.  2:pp.  3-43 

The  underlying  techniquea  of  taak  analyala,  acaling 

methodology,  and  multi -attribute  utility  (MAU)  theory  have 

been  integrated  into  one  comprehenaive  methodology. 

MOAT  ayatematlcally  atructurea  operator  taaka  in  accordance 
with  miaaion  needa  and  then  aaaeaaea  the  operability  of 
theae  taaka  through  conjoint  maaaurement.  All  aaaeaament a 
are  then  integrated  within  rulea  eatabliahed  through  MAU 
theory.  The  output  of  MOAT  ia  quantitative  information 
about  the  operability  of  an  entire  ayatem,  auch  aa  a 
fighter  or  attack  aircraft;  the  operability  of  apecific 
aubayatema  auch  aa  radar,  communicationa,  or  navigation, 
and;  finally,  tha  operability  of  each  taak  performed  during 
a  miaaion  phaae.  In  abort,  MOAT  meaaurea  aubayatem  and/or 
ayatem  goal  attainment. CRef.  2:p.  43 


Having  briefly  described  both  Integration  Analyaia  and 


MOAT#  parte  of  each  were  combined  in  this  theaia  to 
provide  a  technique  that  ia  well  suited  to  a  layout 
iaproveaent  application.  Proa  MOAT  the  techniquee  to  aasess 
man-system  compatibility  <i.e.#  Task  Analysis  and 
Operability  Analysis)  ware  used.  This  was  considered  in  a 
larger  context  in  order  to  aasaaa  man  as  a  team  rather  than 
as  an  individual.  Operability  Analysis  consists  of  two 
parts:  multi 'attribute  utility  (MAU)  theory  (to  be  discussed 
later) #  and  scaling  theory.  The  use  of  questionnaires  and 
Link  Analysis  came  from  Integration  Analysis.  Note  that 
both  Integration  Analysis  and  MOAT  contain  Task  Analysis. 
The  questionnaires  serve  two  purposes.  First.  they  focus 
attention  on  the  problem  areas  of  the  design/arrangement. 
Secondly,  the  completed  questionnaires  support  assessment  of 
the  layout  effectiveness.  Finally,  the  questionnaires  form 
a  link  between  the  various  analyses  and  operator  inputs. 

MOAT  was  designed  to  assess  man-system  compatibility. 
The  original  MOAT  used  a  construct  that  embodied  the  three 
most  important  divisions  of  the  man-system  compatibility-- 
man.  system.  and  mission  and  how  all  three  interact  during 
mission  accomplishment.  The  difference  considered  here  lies 
in  the  evaluation  of  a  team  of  operators  rather  than  oust 
one  man  and  the  fact  that  a  group  of  subsystems  arranged  in 
a  particular  manner  ia  used  to  determine  the  operability. 
Hence.  the  three  moat  important  divisions  of  the  man-system 


compatibility  ara  now  tha  team,  system,  and  aiaaion.  Aa  can 
ba  aaaily  Man,  tha  baaic  praaiaa  of  MOAT  ia  unchanged  in 
that  tha  man-system  coapatibility  ia  atill  baing  avaluatad. 
In  abort,  ia  tha  arrangaaant  of  equipment  in  a  apaca  “uaar 
friendly"?  Nota  that  avan  though  an  arrangaaant  of 
equipment  within  a  work  apaca  ia  baing  apacifieally 
addressed,  aan-ayataa  coapatibility  aubauaaa  tha  aquipaant 
arrangaaant  within  tha  work  apaca.  Tha  baaic  contantion  is 
that  tha  bast  oparators  and  baat  subsystems  in  a  poorly 
designed  apace  may  be  less  effective  than  operators  with 
less  ability  and  a  leas  capable  system  that  is  in  a  space 
that  is  well  designed  for  mission  accomplishment. 

MOAT  uses  the  term  operability  to  reflect  how  the  man. 

systea.  and  mission  interact  during  mission  accomplishment. 

By  definition,  operability  reflecta  (1)  the  amount  of 
effort  required  by  the  operator  in  task  accomplishment,  (2) 
the  degree  of  subsystem  technical  ef fectiveneas  in  aiding 
the  operator  in  task  accomplishment,  and  (3)  how  important 
the  task  is  for  mission  success. CRaf.  2:p.  191 

This  can  be  redefined  slightly  to  indicate  (1)  the  amount  of 

effort  required  by  the  operators  (team)  in  task 

accomplishment  (task  difficulty),  (2)  the  degree  that 

equipment  arrangement  aids  the  operators  (team)  in  task 

accomplishment  (arrangement  effectiveness) ,  and  (3)  how 

important  the  task  is  for  mission  success  (task 


criticality) . 


».  TASK  ANALYSIS 


The  purpoM  /or  Took  Analysi*  ia  to  determine  those 
took#  and  aubtaaka  noodad  to  successfully  par/ora  tha 
alaaion  o /  tha  Module.  Without  apacifying  tha  taaka 
par/oraad  within  tha  EW  Modula,  it  would  ba  difficult,  if 
not  iapoaaibla,  to  dataraina  layout  affactivanaas. 
taak/aubtaak  idantif ication  foraa  tha  baaia  of  both  tha  Link 
Analyaia  that  ia  diacuaaad  latar  and  tha  MOAT  technique  of 
Oparability  Analyaia.  Each  taak  and  aubtaak  that  operators 
parfora  will  ba  axaalnad  and  fit  into  tha  larger  picture  of 
aodula  niaaion.  Tha  effort  within  tha  EW  Modula  can  ba  shown 
to  ba  divided  hierarchically:  Modula  .Mission,  operator 
tasks,  and  operator  aubtaaka.  This  hierarchy  ia  divided  in 
tha  following  Manner:  tha  aggregate  of  the  subtasks 
coaprisaa  tha  individual  taak  and  tha  aggregate  of  the 
individual  taaka  coMpriaea  tha  niaaion. 

The  Mission  of  the  EW  Module  ia  to  conduct  Electronic 
Warfare  which  includes  Electronic  Warfare  Support  Measures 
(ESM)  and  Electronic  Warfare  Countar-Maaauras  (ECM) .  This 
entails  attempting  to  deny  any  potential  enemy  the 
exploitation  of  the  electromagnetic  spectrum  while 
preserving  it  for  our  own  use.  The  EW  Supervisor  (EWS)  is 
responsible  for  providing  timely  evaluated  EW  information, 
EW  data,  and  EW  control  (to  the  rest  of  the  battle  group 
whan  so  designated  as  EW  Control  Ship) .  This  is  accomplished 


by  three  operators  and  three  work  stations  (WLR-1 


SLQ-17 


and  NTDS) .  Tha  taaka  and  subtaaks  that  ara  performed  within 
tha  workapaca  ara  liatad  in  Tabla  1.  Whila  tha  modula 
aiaaion  dalinaataa  tha  ovarall  raaponaibility  for  tha 
module,  tha  oparator  taaka  ara  tha  firat  major  aubdivision. 
Thaaa  ara  tha  taaka  that  aach  oparator  muat  accomplish  at 
hia  workatation  in  ordar  to  contributa  to  miaaion 
accoapliahmant .  Tha  oparator  aubtaak  ia  a  furthar  division 
of  tha  oparator  taaka.  Thaaa  aggragata  togathar  for 
ESM/ECM.  Thaaa  ara  liatad  in  Tabla  1  and  ware  drawn  from 
varioua  aourcaa  and  confirmed  by  the  EW  Module  personnel. 
Each  workatation  and,  therefore,  aach  oparator  has  a  piece 
of  tha  “puzzle**  and  only  by  putting  them  together  can  any 
aanaa  be  made  out  of  tha  parta.  In  thia  caae,  as  so  many 
othara,  tha  whole  will  be  greater  than  the  sum  of  the  parts. 

Note  in  Tabla  1  that  there  ara  actually  five  positions 
liatad:  EW  Supervisor,  WLR-1  oparator,  SLQ-17  operator,  NTDS 
oparator,  and  Statua  Board  keeper.  During  normal  steaming 
conditions,  one  of  tha  three  position  operators  is  also  the 
EW  Supervisor  and,  therefore,  he  has  a  dual  role  to  play. 
Additionally,  .  there  ia  no  Statua  Board  keeper  during  normal 
steaming  watches.  During  Condition  One,  General  Quarters, 
the  Module  ia  manned  with  five  people.  Therefore,  the  Task 
Analysis  considered  the  more  complicated  situation  of 


General  Quarters. 


TABLE  1.  OPERATOR  TASKS  AND  SUBTASKS 


EW  SuprvUor 

Talk :  1.1  Diraet  ESN  March 
Subtaaka 

1.1.1  A aalgn  aaareh  paraaatara  to  SLQ-17 

1.1.2  Aaalgn  March  paraaatara  to  WLR-1 

1.1.3  Aaalgn  ESN  aanaor  raport  raaponaibllitlaa  --  own  ahip 

1.1.4  AMlgn  ESN  aanaor  raport  raaponaibllitlaa  —  forca 
1.1.9  Inltlata  aanual  ID  raquaat  -  ahip 

1.1.6  Inltlata  aanual  ID  raquaat  -  forca 

1.1.7  Monitor  autoaatic  corralationa/aaaoclationa,  (SLQ-17) 

Taak:  1.2  Raport/DlaMainata  EW  Inforaation 
Subtaaka 

1.2.1  Raport  avaluatad  EW  inforaation 

1.2.2  Provida  EW  racoaaandationa 

1.2.3  Updata  atatua  board  naar  NTDS  conaola 

1.2.4  Briaf /dabriaf  aabarkad  Airwinga 

1.2.5  Navigation  by  paaaiva  EW 

Taak:  1.3  Countar  Hoatlla  Environaant 
Subtaaka 

1.3.1  Proaulgation  of  ECM  aaployaant  critaria 
NTDS  Onarator 

Taak:  2.1  Collact  and  antar  EW  data  into  NTDS 
SMfrfcMlS* 

2.1.1  Entar  aanual  ID  inforaation  into  NTDS 

2.1.2  Entar  aanual  ESM/NTDS  track  aaaociationa 

2.1.3  Parfora  triangulation  of  ESM  baaring  linaa 

2.1.4  Entar  EW  fixaa 

2.1.5  Adviaa  oparatora  of  baaring  raaolutlon 

2.1.6  Evaluata  axtarnally  raportad  ESM  baaringa 

Taak:  2.2  Raport/Diaaaainata  EW  Information 

2.2.1  Raport  avaluatad  EW  inforaation 

2.2.2  Updata  atatua  board  naar  conaola 

Taak:  3.1  Conduct  ESM  Saarch 

Safe&asls 

3.1.1  Monitor  autoaatic  corralationa/aaaoclationa 

3.1.2  Eatabllah  oparatlng  aodaa  of  SLQ-17  (ESM) 

3.1.3  Entar  dataction  and  raaponaa  paraaatara  (ESM/ECM) 

3.1.4  Monitor  anvironaant  on  NTDS  conaola 

3.1.5  Evaluata  diaplayad  data 


TABLE  1.  OPERATOR  TASKS  AND  SUBTASKS  (continued) 


Taak :  3.2  Raport/Diaaaminata  EW  Information 

aute&aafc. 

3.2.1  Raport  avaluatad  EW  information 

3.2.2  Provida  ESM/ECM  data  to  NTDS 

3.2.3  Monitor  antry  of  EW  data  into  NTDS 

3.2.4  Updata  atatua  board 

Taak:  3.3  Countar  Hoatila  Environmant 

SuDtfllK 

3.3.1  Engaga  targata  with  ECM 

3.3.2  Eatabliah  ECM  oparating  Modaa 

3.3.3  Aaaiat  in  promulgation  of  ECM  amploymant  critaria 
WLR-1  Qparator 

Taak :  4.1  Conduct  ESM  Saarch 
Subtaaka 

4.1.1  Saarch  aaaignad  banda 

4.1.2  Analyza  intarcaptad  aignala 

4.1.3  Chack  intarcapta  for  imagaa/harmonica 

4.1.4  Accurataly  DF  intarcaptad  aignala 

4.1.5  Aaaiat  in  avaluating  ECM 

Taak;  4.2  Raport/Diaaaminata  EW  Information 
Subtaaka 

4.2.1  Provida  ESM  data  to  NTDS 

4.2.2  Raport  avaluatad  EW  information 

4.2.3  Updata  atatua  board  naar  poaition 

4.2.4  Log  all  intarcapta 

Taak;  4.3  EMCON 

4.3.1  Monitor  EMCON 

4.3.2  Raport  violationa  of  EMCON 

4.3.3  Log  violationa  of  EMCON 

4.3.4  Monitor  MUTE 


C.  LINK  ANALYSIS 


Link  Analysis  is  a  technique  that  will  provida  the 
information  naadad  to  produca  an  accaptabla  arrangement  of 
nan  and  nachinas  in  a  systan  CRaf .  3:p.  2043 .  Tha  idaa  is 
that  tha  **bast  arrangaaant”  can  ba  found  only  by  optimizing 
diffarant  typas  of  links  that  ara  important  in  tha 
particular  systam  baing  dasignad.  By  way  of  definition,  a 
link  is  a  connection  between  (a)  an  operator  and  a  machine 
or  <b)  two  operators  CRaf.  3:p.  2043 .  These  links  may  be 
visual  (such  as  an  instrument  scan) ,  functional  or  manual 
(hand  to  control),  or  verbal  (communications). 
Inaff iciancias  ara  present  whan  links  are  comparatively 
long,  crossing  one  another,  blocked,  or  outside  optimal 
visual  or  reach  envelopes.  Tha  links  are  produced  from  the 
task  analysis  and  illustrate  all  tha  operator-required 
functional,  visual,  and  communication  tasks.  Link  Analysis 
can  ba  applied  to  all  scenarios  involved  during  oil 
operational  and  amargancy  conditions  CRaf.  3:p.  205  and 
Ref .43 .  Link  Analyses  ara  normally  of  two  typas:  panel 
layout  and  tactical  compartment  or  multiple  operator  work 
area.  With  tha  development  and  procurement  of  individual 
subsystems  (i.e.,  WLR-1,  SLfl-17,  etc.)  a  certain  amount  of 
panel  layout  link  analysis  has  bean  dona.  However,  little 
if  any  has  bean  dona  on  tha  combination  of  systems  arranged 
in  a  workspace  (in  this  example,  tha  EW  Module) .  Hence,  tha 


Link  Analysis  will  nscaaaarily  bs  of  the  lattsr  typs  (i.a., 
sul tipis  opsrstor  work  sru)  . 

Tha  sultipls  opsrstor  work  srss  typs  of  link  analysis  is 
dspsndsnt  on  ths  corrslstion  matrix.  Baginning  with  tha 
corralation  matrix  and  an  araa  layout,  all  intaractions 
(links)  raquirad  to  parform  a  particular  functional  task  are 
examined  in  terms  of  tha  frequency  with  which  they  occur  and 
their  criticality.  If  the  criticality  is  assigned  a 
numerical  value,  it  may  be  multiplied  by  the  frequency  in 
order  to  obtain  a  weighted  link  value.  The  work  area  is 
overlaid  with  the  weighted  links  permitting  a  picture  of  all 
the  interactions  taking  place  within  the  system  being 
analyzed.  The  system  design  csn  then  be  modified  to  shorten 
the  distance  between  the  workstations  that  are  connected  by 
the  weighted  links  CRef.  53. 

Figure  3  contains  the  correlation  matrix  for  the  EW 
Module  in  CARL  VINSON.  A  correlation  matrix  is  a  figure 
that  provides  an  indication  of  the  links  between  two 
operators,  positions,  or  between  an  operator  and  a  position. 
Usually  a  criticality  associated  with  the  particular  links 
is  included  in  the  matrix.  In  Figure  3,  only  the  links  of 
interest  are  listed.  The  figure  is  read  by  selecting  the 
two  entries  for  which  links  are  desired  and  reading 
diagonally  down  from  the  top  one  and  diagonally  up  from  the 
bottom  one  until  the  intersecting  diamond  is  reached.  The 
diamond  contains  both  the  particular  links  between  the  two 


Internal  Co 


Figure  3.  Link  Correlation  Matrix 


intriu  and  the  criticality  of  thoaa  linka.  Tha  notation 
along  tha  right  aida  of  tha  figura  indicataa  what  links  ara 
batwaan  aach  coaponant  of  tha  aodula.  A  aaparata  notation 
providaa  an  indication  of  tha  criticality  of  that  particular 
link:  aoat  critical,  critical,  and  laaat  critical.  Tha  moat 
critical  link  is  ona  that  is  abaolutaly  aaaantial  to 
accoapliah  tha  aiaaion.  A  critical  link  ia  ona  that 
pravanta  aavara  dagradation  in  tha  accoapliahaant  of  tha 
aiaaion  and  tha  laaat  critical  link  haa  a  aaall  impact.  It 
raaaina  only  to  multiply  tha  linka  by  a  fraquancy  of 
oparation  to  obtain  a  Waightad  Link  Valua.  Tha  Weighted 
Link  Valua  will  ba  obtainad  from  tha  raaults  of  the  Link 
Analysis  Questionnaire. 

Tha  type  of  linkt  ara  communication,  visual,  and  manual. 
Tha  communication  linka  ara  further  subdivided  into  internal 
and  external.  Tha  internal  communication  links  are  the 
voice  interaction  batwaan  operators  while  the  external 
communication  linka  ara  thoaa  voice  and/or  electronic  links 
with  other  modules,  parsons,  and/or  platforms.  Tha  visual 
linka  can  also  ba  divided  into  internal  and  external.  The 
internal  ara  concerned  aolaly  with  thoaa  linka  between  the 
operator  and  his  equipment  or  console.  Tha  external  are 
thoaa  batwaan  tha  operator  and  other  consoles  and/or 
equipment.  Similarly,  tha  manual  links,  internal  and 
external , 


console 


ara  batwaan  tha  operator  and  hia  equipment  or 
and  tha  operator  and  other  equipment  and/or 


consoles*  respectively .  Sines  we  hsvs  already  considered 
that  link  analysis  nay  have  been  done  on  individual  systems 
and  our  concern  is  for  the  multiple  operator  work  area*  we 
will  not  be  concerned  with  the  internal  visual  and  internal 
manual  links.  This  leaves  just  four  links  analyses  to  be 
done;  the  internal  and  external  communications*  the  external 
visual*  and  external  manual  links. 

When  the  internal  communication  links  are  considered*  it 
is  noted  that  there  are  links  between  the  EW  Supervisor  and 
both  the  WLR-1  and  tha  SLQ-17  operators  to  promulgate 
orders.  Next*  there  are  links  back  to  the  EW  Supervisor 
when  one  or  the  other  have  found  either  the  signal  of 
interest  or  something  else  that  nay  be  of  interest.  There 
are  also  links  between  the  WLR-1  operator  and  the  SLQ-17 
operator.  This  last  may  be  queries  for  information  about 
their  particular  equipment  set  up  or  the  passing  of 
information  to  directing  the  other's  search.  The  links 
between  all  three  may  be  in  the  form  of  equipment  status  or 
failure.  The  internal  communication  links  are  shown  in 
Figure  4. 

The  external  communications  involve  primarily  the  EW 
Supervisor  with  any  of  the  following:  TAO*  INTEL  (the 
Intelligence  center)*  SURFACE*  AIR*  Trackers*  ASW  (Anti- 
Submarine  Warfare)  Module*  SSES  (Ships  Signal  Exploitation 
Space)*  FLAG*  FWC  (Force  Weapons  Coordinator)*  or  SWC 


(Ship's  Weapons  Coordinator) 


gure  4.  Internal  Communications  Links 


Ths  external  vleual  links  involve  all  o£  the  operators 
viewing  the  statue  boards  and  each  other's  workstation. 
These  external  visual  links  ara  important  for  the  additional 
information  they  provide  to  the  operator.  The  visual 
reference  to  the  other  workstation  builds  an  internal 
working  model  of  the  environment  within  the  operator's  mind 
thus  enabling  him  to  more  quickly  fit  new  data  into  the 
tactical  picture  and  anticipate  subsequent  events.  Without 
this  interaction,  effective  Electronic  Warfare  control  can 
not  be  attained. 

Figures  5  and  6  show  the  external  visual  links.  Figure  5 
shows  visual  links  between  operators  and  Figure  6  indicates 
visual  links  between  the  operators  and  the  status  boards 
(there  are  only  two  status  boards  that  can  be  clearly  seen) . 

The  final  link  that  will  be  considered  is  the  external 
manual  link.  Although  there  is  no  requirement  for  an 
operator  to  control  more  than  one  workstation,  there  are 
some  external  manual  links  that  must  be  addressed.  For 
example,  all  those  equipments  that  are  part  of  the  EW  Module 
for  which  there  is  no  manning  authorized  will  fall  under 
this  category.  The  AN/SSQ-82  MUTE  is  a  prime  example.  MUTE 
is  a  monitor  device  that  requires  no  manning  and  only 
cursory  glances  to  ensure  that  it  is  functioning  correctly. 
When  adjustment  is  needed  due  to  Force  or  ship  EMCON 
changes,  an  external  manual  link  for  one  of  the  operators 
takes  place.  Another  example  is  the  SLQ-17  UYK-20  computer. 


External  Visual  Links 


External  Visual  Links  -  Status  Boards 


This  equipment  ie  not  ad 3 scent  to  the  SLQ-17  and  one  of  the 


operetore  (normally  the  SLQ-17  operator)  may  need  to  reload, 
reboot,  or  reconfigure  the  system  in  the  event  of  a  casualty 
or  normal  operations.  Figure  7  shows  the  external  manual 
links. 

D.  OPERABILITY  ANALYSIS 

In  the  introduction  to  this  section,  it  was  stated  that 
Operability  Analysis  was  comprised  of  MAU  and  scaling 
theory.  MAU  ie  a  Bayeaian-oriented  decision-making 
paradigm.  There  are  three  major  aspects  of  the  MAU  model 
which  are  particularly  important  to  this  application.  First, 
the  basic  structure  principle  in  MAU  is  hierarchical 
decomposition.  The  mission  is  broken  down  into  hierarchical 
grouping  of  tasks  and  subtasks.  The  model  provides  the 
structure  and  rules  necessary  to  investigate  and  integrate 
the  interrelationships  of  all  these  tasks  and  subtasks. 
Second,  the  definition  of  utility  used  in  the  MAU  model 
allows  for  the  optimum  evaluation  of  alternatives  which  is 
dependent  upon  the  selection  of  a  single  criterion.  This 
means  that  multidimensional  outcomes  must  be  transformed 
into  a  single  figure  of  merit  such  as  utility,  system  worth, 
system  effectiveness,  or,  as  in  this  application, 
operability.  Third,  a  scaling  of  the  selected  criterion. 
The  scaling  methodology  used  in  this  application,  as  in 
MOAT,  wss  conjoint  measurement. 


External  Manual  Links 


Recall  that,  operability  can  be  viewed  ae  a  function  of 


taek  criticality i 


operator  workload , 


and  apace 


ef f ecti veneee .  Therefore,  when  considering  each  task  fro*  a 
operability  standpoint,  each  task  that  is  performed  has  soma 
combination  of  these  three  dimensions.  There  is  a 
difficulty  in  assessing  the  degree  of  each  attribute  and 
combining  them  into  a  meaningful  measure  of  operability. 


Since  this  can  not  be 


id  directly  by  objective 


methods,  the  scaling  methodology  of  conjoint  measurement  was 
devised  to  assess  space  operability  subjectively.  The 
problem  of  scaling  tasks  in  dimensions  of  criticality, 
frequency,  and  system  effectiveness  has  been  successfully 
solved  by  using  objectively  anchored  rating  scales 
CRef.  2:p.  201.  Therefore,  a  similar  rating  scale  procedure 
seemed  suitable  in  this  instance. 

The  major  difficulties  involved  with  this  approach  are 
those  of  measuring  the  degree  of  operator  effort  (or  watch 
section  effort)  and  the  layout  ef fectiveness.  This  was  to 
be  expected,  however,  since  not  only  were  these  different 
from  any  known  previous  study  but  also  involved 
interactions  on  a  higher  scale  than  that  experienced  before. 
There  is  a  aubatantlal  correlation  in  rating  of  task 
difficulty  and  subsystem  effectiveness.  The  attempt  to 
solve  the  rating  scale  problem  is  accomplished  by  dividing 
it  into  two  separate  ratings.  On  the  F/A-18  program  it  was 
desirable  to  have  two  ratings;  one  with  respect  to  pilot 


workload 


<PVI>  and  ona  with  raapact  to  tha  tachnlcal 


aff act iveneae  (TE)  of  tha  aubayataa  CRaf.  2:p.  203.  A 
alallar  approach  la  attaaptad  hara  with  oparator  workload 
<0%»>  (It  la  aaauaad  that  thla  can  ba  dlractly  tranalatad 
into  watch  aaction  workload)  and  apaca  aff actlvanaaa  (SE) . 
Thla  application  of  Oparability  Analyaia  la  concarnad  with 
aaaaaaing  tha  taak  criticality,  tha  oparator  workload,  and 
tha  apaca  aff actlvanaaa  of  a  aodula  layout.  All  ara  valuaa 
on  an  ordinal  acala.  Two  of  thaaa,  apaca  aff actlvanaaa  and 
oparator  workload,  naad  to  ba  upgradad,  through  aoaa 
aathodology,  to  an  intarval  acala  in  ordar  to  aggragata  thaa 
ovar  all  taaka  to  achiava  an  ovarall  aaaaura  of  Hodula 
oparability.  To  thia  and,  conjoint  aaaauramant  and  ita 
aaaociatad  acaling  procaduraa  aaemed  auitabla  for  a 
tranaformation  to  tha  daairad  unldiaanaional  intarval  laval 
acala.  It  ia  hara  that  tha  dalta  aathod  waa  aaployad.  What 
conjoint  aaaauramant  and  tha  dalta  aathod  do  ia  allow 
aaparata  rating  of  0W  and  SE  (daapita  thair  mutual 
dapandancy)  to  ba  obtainad  and  ba  combinad  in  auch  a  manner 
that  a  one-dimenaional  acala,  having  intarval  propertiea,  ia 
created.  Thia  acala  might  juat  aa  wall  ba  called  the 
combined  QW-SE  acale.  Rating  on  thia  acala  can  then  be 
plugged  into  tha  MAU  modal .  An  aaaaaamant  of  tha 
affactivanaaa  of  tha  EW  Hodula  layout  with  raapact  to  a 
certain  aubtaak  can  ba  determined.  Aggregated  together. 


thaae  will  provide  an  indication  of  the  overall 
of  the  layout  upon  aiaaion  accoapliahaant . 


E.  QUESTIONNAIRES 

Link  Analysis,  Taak  Analysis,  and  Operability  Analysis 
will  ba  coapletad  by  a  seriaa  of  two  queationnairea.  Tha 
first  questionnaire  <the  Link  Analyaia  Questionnaire)  was 
targettad  at  Link  Analyaia  and  providad  tha  fraquancy 
coaponent  for  a  coaplatad  Link  Analysis.  The  second 
questionnaire  (the  Operator  Subtaak  Queationnaira)  confirmed 
tha  Task  Analysis  that  want  into  building  it  and  also 
providad  the  raw  data  needed  to  perform  tha  Operability 
Analysis.  From  the  Operability  Analysis  came  the  assessment 
of  the  Total  Module  Operability  (TMO) . 

The  queationnairea,  as  shown  in  Appendix  A,  ware 
designed  to  do  two  things.  First,  the  Link  Analysis 
Questionnaire  gave  a  general  idea  of  the  type  and  degree  of 
deficiencies  in  the  apace  in  terms  of  link  deficiencies. 
Second,  it  focused  the  operators'  attention  on  the 
arrangement  of  the  work  space  and  any  deficiencies  that  were 
there.  It  was  hoped  that  the  questions  brought  into  sharp 
relief  the  difficulties  for  which  the  operator  unconsciously 
compensates  during  the  mission.  By  highlighting  these 
deficiencies  through  the  Link  Analysis  sarias  of  questions, 
the  detailing  of  them  in  tha  Operator  Subtask 


Questionnaires,  hopefully,  provided  good  human  engineering 


data  with  which  to  avaluata  tha  Modula  Operability  of  tha  EW 
Hodula. 

Tha  Link  Analysis  Quastionnaira  was  dasignad  to  determine 
tha  frequency  of  tha  various  links  in  tha  EW  Hodula. 
Coabinad  with  tha  eorralation  satrix  that  indicates 
criticality,  this  quastionnaira  datarainad  tha  waights  of 
tha  various  links.  This  waighting  indicatad  the  most 
heavily  used  links.  This,  in  turn,  can  focus  attention  on 
deficiencies  in  these  links.  A  possible  exaaple  of  this 
eight  well  be  the  abnoraally  long  internal  comaunication 
link  between  the  NTDS  console  and  the  WLR-1  position. 

Tha  Operator  Subtask  Questionnaire  contained  a  section 
requiring  an  aasessaant  for  each  of  the  forty-nine  subtasks 
delineated  in  Table  1.  The  subtasks  were  drawn  from  a 
variety  of  sources  (including  USS  CARL  VINSON  Combat 
Direction  Center  doctrine)  and  verified  by  the  EW  operators 
prior  to  the  administration  of  the  questionnaire.  This 
assessment  was  the  culmination  of  the  Task  Analysis  and  the 
beginning  of  the  Operability  Analysis.  In  the  Subtask 
Assessment,  each  subtask  on  the  questionnaire  was  evaluated 
by  the  EW  operators  with  regards  to  Operator  Workload,  Space 
Ef fectiveness,  and  Criticality  of  the  Subtask  to  the  overall 
mission  accomplishment. 

The  Operator  Subtask  Questionnaire  also  contained  the 
Ranking  Matrix,  where  combinations  of  the  various  degrees  of 
Operator  Workload  (OW)  and  Space  Ef f ectiveness  (SE)  were 


TABLE  2.  RANKING  MATRIX 


In  tha  following  matrix,  tha  blocks  ara  ranked  from 
beat  to  worst  <1  to  16).  The  lowest  numbered  block  is  the 
Intersection  of  the  best  of  the  rows  and  columns.  The  number 
two  block  is  next  best,  and  so  on.  Note  the  arrows  and  the 
phrases  associated  with  them.  Design  means  the  design  of 
the  layout  or  arrangement. 
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ranked 


Thia  was  of  the  utmost  importance  ainca  the 


conjoint  aaaauraaant  of  tha  aubtaak  aaaaaaaant  and  Module 
oparability  depended  upon  it.  '/able  2  containa  a  blank 
Ranking  Matrix.  Tha  EW  pereonnel  ware  aakad  to  rank  the 
intaraactiona  from  beat  to  worst  for  tha  ‘‘typical"  aubtaak. 
It  waa  aaauaad  that  tha  rank  order  for  tha  aatrix  would  vary 
little  from  aubtaak  to  aubtaak.  Halaa  found  thia  to  be  true 
CRaf .  2:p.  343 .  Thia  may  have  bean  tha  aoat  difficult  part 
of  tha  questionnaire  and  tha  EW  oparatora  were  forced  to 
draw  upon  all  their  knowledge  and  previoua  experience  in 
order  to  produce  a  rank  order  that  waa  meaningful  and 
replicable.  Thia  matrix,  tha  intersection  of  two  ordinal 
acalea  COW  and  SE> ,  ia  part  of  conjoint  measurement.  The 
Ranking  Matrix  waa  expanded  and  an  interval  scale 
constructed  via  a  linear  expansion  know  as  the  delta  method. 
Thia  resulted  in  an  Interval  scaling  from  0  to  100  and  was 
uaed  to  evaluate  the  total  Module  oparability.  The  delta 
method  of  converting  two  of  theae  ordinal  scales  to  an 
interval  scale  ia  described  in  Appendix  C. 

Using  thia  interval  scale,  the  intersection  of  any 
particular  set  of  Operator  Workload  and  Space  Ef f ectiveneas 
values  on  the  returned  Operator  Subtask  Questionnaire  gives 
a  predetermined  Operability  Value  between  0  and  100.  An 
Operator  Workload  value  between  one  and  four  inclusive 
served  to  identify  a  column  while  a  Space  Ef f ectiveneas 
value  identified  a  row.  The  intersection  of  the  row  with 


the  column  indicated  tha  aaaaaaaant  of  that  particular 
aubtaak  by  an  operator.  For  avary  aubtaak,  thia  Operability 
Value  was  obtained  for  each  rater  and  the  naan  and  atandard 
deviation  were  calculated.  Thia  mean  value  rapreaented  the 
Operability  Value  for  that  aubtaak. 

The  remaining  ordinal  acale  ia  that  of  the  Criticality. 
There  was  no  attempt  to  convert  thia  to  an  interval  acale. 
Although  operators'  akilla  might  vary,  cauaing  aignificant 
deviationa  in  the  ratings  from  rater  to  rater,  there  should 
be  only  one  atandard  for  the  criticality  of  a  aubtaak  aa  it 
relates  to  mission  accomplishment.  This  single  measure  of 
criticality  was  taken  to  be  the  mean  of  the  criticality 
ratings.  The  Operability  Value  was  multiplied  by  the 
criticality  resulting  in  a  Weighted  Operability  Value.  A 
Weighted  Deficit  Value  was  computed  aa  (100  -  Operability 
Value)  multiplied  by  the  Criticality  of  the  aubtaak. 
Whereas  the  Weighted  Operability  Value  will  give  an 
indication  of  the  ''goodness**  of  the  layout  for  a  particular 
aubtaak,  the  Weighted  Deficit  Value  gives  an  indication  of 
how  much  improvement  ia  required  to  optimize  Module 
Operability  for  a  particular  aubtaak. 

The  Link  Analysis  Questionnaire  was  given  approximately 
one  week  before  the  Operator  Subtaak  Questionnaire.  It  was 


hoped  that  the  brief  exposure  to  the  first  questionnaire 
increased  the  accuracy  of  the  second. 


iv.  BE9VUTS 


To  tost,  this  application  ,•  a  auitabls  platform  was 
required.  Ths  Electronic  Warfars  Module  on  an  U.S.  Navy 
aircraft  carrisr  was  aslsctsd.  Ths  particular  ship,  USS 
CARL  VINSON  <CVN-70> ,  was  chossn  for  thrss  reasons: 
availability,  accsssibility ,  and  familiarity.  CARL  VINSON 
had  just  rsturnsd  from  a  ssvsn  month  cruiss  and  was  in  a 
stand-down  psriod  and,  so,  available.  The  ship's  homeport, 
Alameda,  Ca.,  was  readily  accessible  for  ths  test.  Finally, 
ths  ship's  layout  was  familiar  enough  to  ths  teat  director 
to  allow  a  minimum  amount  of  time  to  be  spent  on  the  ship 
and,  therefore,  lessen  ths  impact  upon  ths  ship's  daily  work 
and  schedule. 

There  were  limitations  to  the  scope  of  testing.  First, 
the  test  was  not  dons  at  sea  which  produced  two  limitations. 
In  regards  to  Link  Analysis,  operator  usage  of  the  various 
links  and  the  associated  frequencies  could  not  be  monitored. 
This  was  considered  to  be  a  major  limitation  in  regards  to 
only  the  Link  Analysis  portion  of  the  test.  The 
compensation  for  this  was  the  Link  Analysis  Questionnaire 
concerning  the  frequency  of  link  usage.  A  minor  limitation 
concerned  the  inability  to  observe  ths  actual  Subtaaka  and 
ascertain  the  criticalities  under  actual  conditions.  This 


was  compensated  by  the  Operator  Subtaak  Questionnaire,  which 
waa  conaidarad  adequate. 

A  further  liaitatlon  waa  the  aaall  number  of  valid 
reaponaea  for  the  queationnairea.  There  were  three  valid 
reaponaea  for  the  Link  Analyaia  Queationnaire,  five  for  the 
Rank  Ordering  portion  of  the  Operator  Subtaak  Queationnaire, 
and  from  five  to  aeven  for  the  Subtaak  Aaaeaaaent  portion  of 
the  Operator  Subtaak  Queationnaire.  While  theae  numbers  are 
email  from  a  atatiatical  point  of  view,  they  can  not  be 
diacounted.  The  limited  aample  aize  ahould  be  an  inducement 
for  further  testing.  Furthermore,  the  sample  size  for  any 
aircraft  carrier  will  never  be  much  greater  than  about 
twelve  due  to  manning  levels.  The  sample  size  was  seven  due 
to  leave  and  various  schools  but  included  the  personnel  with 
the  moat  experience.  In  may  be  argued  that  not  testing 
other  platforms  is  a  limitation.  However,  since  no  two  EW 
Modules  on  U.S.  aircraft  carriers  are  alike,  the  lack  of 
multiple  testing  ia  a  moot  question. 

The  test  was  conducted  in  the  EW  Module  of  USS  CARL 
VINSON  (CVN-70) .  The  Module  was  used  so  that  the  personnel 
could  refresh  their  memory  with  regards  to  the  layout  as 
they  evaluated  the  subtaska  in  relation  to  the  layout. 

The  guidance  given  to  the  EW  personnel  before  and  during 
the  teat  stressed  that  they  could  ask  any  question  they 
wished  of  anyone  they  wished.  They  were  encouraged  to 


confer  with  each  other  about  the  workload,  effectiveness, 
and  criticality. 

A.  LINK  ANALYSIS 

The  results  of  the  Link  Analysis  were  taken  from  the 
Link  Analysis  figures  and  from  the  Link  Analysis 
Questionnaire.  The  questionnaire  was  produced  from  the  Link 
Analysis  figures  and  the  Task  Analysis  in  order  to  determine 
the  frequency  that  these  links  were  used.  The  EW  operators 
on  USS  CARL  VINSON  were  asked  to  estimate  how  many  times 
during  a  standard  eight  <8)  hour  watch  they  utilized  the 
links.  The  Link  Analysis  Questionnaire  is  listed  in 

Appendix  A  and  the  results  of  the  Link  Analysis  is  shown  in 
Table  3. 

1.  Link  Analysis  Figures 

The  most  critical  links  were  assessed  to  be  the 
communication  links  between  operators  and  the  visual  links 
between  positions.  The  criticality  of  the  links  were  chosen 
to  reflect  mission  accomplishment  and  the  frequency  of  usage 
confirmed  the  criticality.  There  were  four  links  considered 
in  the  Link  Analysis:  internal  communication,  external 
communication,  external  visual,  and  external  manual.  Of 
these  four,  the  two  most  important  links  are  the  internal 
communications  and  external  visual.  This  is  because  the 


external  communication  will  generally  involve  only  one 
operator  <the  EW  Supervisor/NTDS  operator)  and  there  is 


TABLE  3.  LINK  ANALYSIS  BY  POSITION 

Weighted 


Position  and  Tasks 

Freouencv 

Link 

Critical itv 

Link  Vai 

UUM  Operators 

1.  Talk/coeaunicate  with  the  SLQ-17  operator? 

16.667 

IC 

3 

50.00 

2.  View  the  presentation  on  the  SLQ-17  console? 

77.333 

EV 

3 

231.99 

3.  Talk/coaeunicate  with  the  NTDS  operator? 

44.  BM 

IC 

3 

132.00 

4.  View  the  presentation  on  the  NTDS  console? 

73.333 

EV 

3 

219.99 

5.  View  the  NTDS  Status  Board  (SB)? 

33.333 

EV 

1 

33.33 

6.  View  the  UUM  Status  Board? 

33.333 

EV 

2 

66.67 

7.  Update  the  UUM  Status  Board? 

1.667 

Eft 

2 

2.34 

8.  Check  (visually)  the  SLQ-17  coeputer? 

31.  BM 

EV 

1 

31.00 

9.  Reboot,  reset,  or  work  with  the  SLQ-17  coeputer? 

1.667 

Eft 

1 

1.67 

IB.  Check  ftUTE? 

3.083 

EV 

2 

6.00 

11.  Change  any  settings  on  KITE? 

B.667 

Eft 

2 

1.34 

SLQ-17  Operator: 


1.  Talk/coeeunicate  with  the  NTDS  operator? 

54.000 

IC 

3 

162.M 

2.  View  the  presentation  on  the  NTDS  console? 

73.333 

EV 

3 

219.99 

3.  Talk/coeeunicate  with  the  UUM  operator? 

71.667 

IC 

3 

215.00 

4.  View  the  presentation  on  the  >0-1  console? 

48.333 

EV 

3 

144.99 

5.  View  the  UUM  Status  Board? 

23.088 

EV 

3 

69.00 

6.  View  the  NTDS  Status  Board? 

31.333 

EV 

3 

93.99 

7.  Update  the  NTDS  Status  Board? 

9.0M 

Eft 

1 

9.08 

8.  Check  (visually)  the  SUM7  coaputer? 

26.667 

EV 

2 

53.34 

9.  Reboot,  reset,  or  work  with  the  SLQ-17  coeputer? 

4.000 

Eft 

3 

12.00 

10.  Check  ftUTE? 

l.OM 

EV 

1 

1.00 

11.  Change  any  settings  on  ftUTE? 

0.667 

Eft 

1 

0.67 

NTDS  Operatcr/EW  Supervisor: 


1.  Talk/coeaunicate  with  the  SLQ-17  operator? 

45.667 

IC 

3 

126.99 

2.  View  the  presentation  on  the  SLQ-17  console? 

65.667 

EV 

3 

197.08 

3.  Talk/coswunicate  with  the  UUM  operator? 

72.333 

IC 

3 

216.99 

4.  View  the  presentation  on  the  UUM  console? 

60.667 

EV 

2 

182.00 

5.  View  the  *0-1  Status  Board? 

45.000 

EV 

3 

135.00 

6.  View  the  NTDS  Status  Boarc? 

48.333 

EV 

3 

144.99 

7,  Update  the  NTDS  Status  Board? 

16.000 

Eft 

3 

•8.00 

8.  Check  (visually)  the  SLQ-17  coaputer? 

34.667 

EV 

1 

34.67 

9.  Reboot,  reset,  or  work  with  the  SLQ-17  coaputer? 

4.000 

Eft 

1 

4.00 

10.  Check  ftUTE? 

6.667 

EV 

1 

6.67 

11.  Change  any  settings  on  JUTE? 

4.000 

Eft 

1 

4.00 

12.  Cooaiunicate  outside  the  ftoduie? 

35.333 

EC 

3 

105.99 

KEY:  IC  -  Internal  Comum  cat  ions;  EC  -  External  Coenunications; 
EV  -  External  Visual;  Eft  -  External  Nanuai 
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own 


little  requirement  for  eenuel  links  outside  of  ones 
position.  An  external  communications  link  example  is  the 
link  between  the  EW  Supervisor/NTDS  operator  and  the 
communications  that  enable  him  to  communicate  outside  the 
Module.  However,  this  requires  that  the  operator  rise  from 
his  seat  to  communicate.  As  a  remedy,  the  NTDS  operator 

uses  a  hand  mike  that  hangs  down  near  his  console.  This  is 

a  partial  solution  because  he  still  needs  to  rise  from  his 
seat  to  select  another  station  on  the  communication  box. 
Additionally,  the  hand  mike  hanging  so  close  to  his  console 
presents  a  clutter  problem. 

Note  that  the  communication  and  visual  link  between 
the  NTDS  operator  <EW  Supervisor)  and  the  WLR-1  operator  is 

the  longest  and  partially  blocked.  The  links  between  the 

WLR-1  operator  and  the  NTDS  operator  and  SLQ-17  operator  are 
long,  allowing  him  to  view  very  little  of  the  environment. 
The  WLR-1  operator's  visual  links  are  very  long  and  the 
parallax  effect  severely  degrades  his  observation.  Note  the 
long  link  lines  between  the  SLQ-17  and  NTDS  positions  and 
the  WLR-1  Status  Board,  and  the  WLR-1  operator  and  the  NTDS 
Status  Board.  Finally,  note  the  very  long  external  visual 
links  to  the  SSQ-82  MUTE  and  that  they  cross.  MUTE  is 
required  to  be  checked  periodically  for  faults  or  changes  in 
the  various  monitor  boxes.  The  distance  is  great  enough 
between  MUTE  and  the  rest  of  the  Module  that  only  the  WLR-1 
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The  frequency  of  the  various  links  wars  determined  by 
the  Link  Analysis  Quastionnaira.  Tha  idaal  way  to  datarsina 
link  fraquancy  is  to  count  tha  actions/link  intaractions 


during  tha  watch. 


Sinca  this  was  not  possible,  the 


quastionnaira  approach  was  chosan.  Tha  Link  Analysis  is 
intendad  harj  to  focus  attention  at  the  links  that  are  used 
most  often.  Tha  fraquancy  of  link  usage  is  multiplied  by 
tha  weight  Ccriticality )  of  tha  link  and  an  indication  of 
its  relative  importance  is  determined. 

When  the  links  associated  with  the  WLR-1  operator 
are  considered,  it  can  be  noted  the  longest  links  are  the 
internal  communication  and  external  visual  links  between  him 
and  tha  SLQ-17  and  NTDS  positions.  These  links  are  also  the 
most  critical  and  the  most  frequently  used.  The  average 
number  of  times  the  operator  tries  to  view  the  NTDS  console 
is  73.333.  Yet  this  console  is  the  furthest  away  (see 
Figures  4  and  5) .  The  WLR-1  operator  communicates  more  with 
the  NTDS  operator  for  two  reasons.  Many  times  the  NTDS 
operator  is  also  the  EW  Supervisor.  The  fullest  picture  of 
the  entire  environment  of  surface,  subsurface,  and  air 
contacts  is  present  on  the  NTDS.  The  other  large  frequency 
usage  is  the  visual  links  for  the  presentation  on  the  SLQ-17 


V.VJV  ■-1’ 


consol*.  This  consols  is  only  slightly  clossr  than  tha  NTDS 
consols. 


In  ths  esss  of  ths  SLQ-17  operator,  tha  first  six 
sntriss  in  Table  3  are  ths  ones  with  tha  greatest 
criticality  and  tha  highest  frequency  of  use.  The  high 
criticality  and  frequency  associated  with  checking  the  SLQ- 
17  coaputer  is  understandable  since  the  SLQ-17  operator  is 
specifically  trained  to  know  what  to  look  for  on  the 
coaputer  face.  Note  that  the  SLQ-17  operator  views  the 
presentation  at  the  NTDS  console  auch  more  than  that  at  the 
WLR-1  position.  It  can  be  seen  from  Figure  5  that  these 
external  visual  links  between  the  SLQ-17  and  the  NTDS  are 
auch  shorter  than  between  SLQ-17  and  the  WLR-1.  At  the  same 
time,  the  SLQ-17  operator  communicates  more  with  the  WLR-1 
operator  than  with  the  NTDS  operator.  This  suggests  that 
the  SLQ-17  operator  gets  a  better  picture  of  the  environment 
from  the  NTDS  but  better  information  concerning  the 
environment  from  the  WLR-1. 

The  NTDS  operator/EW  Supervisor  are  combined  because 
many  times  the  EW  Supervisor  will  man  the  NTDS  console  for  a 
major  portion  of  the  watch.  This  is  necessary  because  all 
the  external  communications  are  at  or  near  the  NTDS  console. 
Note  the  large  frequency  and  high  criticality  associated 
with  communications  outside  the  Nodule  (external 
communications  link) .  There  appears  to  be  a  reversal  of 


Interaction  between  the  NTDS  operator/EW  Supervisor  and  the 


WLR-1  and  SLQ-17  poaitiona.  Ha  views  tha  SLQ-17  conaola 
aora  than  comnunicataa  with  tha  oparator  but  talka  mora  to 
tha  WLR-1  oparator  than  viawa  tha  WLR-1  diaplaya.  Racall 
from  Figures  4  and  5  that  both  tha  intarnal  communications 
and  tha  axtarnal  viaual  linka  batwaan  NTDS  and  WLR-1  art 
vary  long.  Additionally,  nota  how  aueh  ha  looka  at  tha  WLR- 
1  Status  Board  avan  though  it  is  tha  furthest  away  (Figura 
6). 

Tha  Link  Analyala  is  important  ainca  it  aarvaa  to 

indicata  which  linka  era  long,  important,  and  possibly 

$ 

overworked .  As  such  it  can  ba  usad  as  a  starting  point  in 
tha  radaaign  of  a  layout  by  showing  which  links  naad  to  ba 
raducad  in  langth.  Tha  Link  Analysis  raaulta  should  also 
support  tha  rasults  of  tha  Oparability  Analysis. 

B.  OPERABILITY  ANALYSIS 

Tha  Oparator  Subtaak  Questionnaire  was  divided  into  two 
parts:  the  Subtask  Assessment  and  the  Ranking  Matrix.  The 

Subtask  Assessment  was  givan  first.  Tha  criteria  for  this 
evaluation  and  the  test  itself  are  givan  in  Appandix  A. 

Tha  second  half  of  tha  Oparator  Subtask  Questionnaire 
was  the  Ranking  Matrix.  All  returned  valid  rankings  <n*5) 
were  entered  into  the  matrix  and  a  mean  determined  for  each 
block  and  the  matrix  numbered  accordingly.  The  standard 
deviation  was  calculated  in  case  of  a  tie.  This  matrix  with 
the  mean  rank  values  and  the  standard  deviation  is 


illustrated  in  Table  4 


Tha  raaultant  rank  matrix  is  shown 


in  Tabls  5. 

Next,  this  rank  ordering  was  converted  to  an  interval 
scale.  This  was  done  by  reversing  the  order  of  the 
numbering  so  that  the  best  of  the  Operator  Workload  and 
Space  Effectiveness  is  #16  and  the  worst  is  #1  (see  Table 
6) .  Using  this  as  a  bass.  the  delta  method  of  linear 
expansion  was  used  to  determine  an  Interval  scale.  See 
Appendix  C  for  a  brief  description,  example  of  the  delta 
method,  and  the  final  work  sheet  for  this  application. 

Table  7  shows  the  result  of  the  delta  method  which  is  the 
desired  interval  scale.  The  results  of  the  delta  method 
were  normalized  by  dividing  all  the  blocks  by  the  highest 
value  in  the  block;  in  this  application  it  was  102.  Table  8 
is  the  normalized  interval  scale  for  this  application. 

The  Operability  Value  was  weighted  (multiplied)  by  the 
mean  assessed  Criticality  of  that  particular  Subtask  to 
derive  the  Weighted  Operability  Value.  The  Weighted 
Operability  Value  has  the  potential  to  range  from  an 
absolute  minimum  of  0  (0x1)  to  an  absolute  maximum  of  500 
(100x5).  The  rrnge  noted  was  14.14  to  41S.67. 

The  Weighted  Deficit  Value  gives  an  indication  of  how 
much  improvement  is  needed  to  optimize  Layout  Effectiveness 
for  a  particular  Subtaak.  The  greatest  Weighted  Deficit 
Value  was  485.00  while  the  least  was  28.09.  The  Weighted 


TABLE  4.  MEAN  RANK  ORDER  AND  STANDARD  DEVIATION 
FOR  EACH  RATING  MATRIX  CELL 


I  Mean  Rank  I 

I  Standard  Deviation  I 

I _ l 


TABLE  5.  RANK  ORDER  OF  OPERATOR  RATING  MATRIX 


TABLE  6.  FINAL  RANK  ORDER  INVERTED  FOR  DELTA  METHOD 


TABLE  7.  DELTA  METHOD  SOLUTION  FOR 
OPERATOR  SUBTASK  RATING  SCALE 


Deficit  Value  can  range  from  900  to  0.  The  larger  the 
number,  the  greater  the  amount  of  improvemant  ia  needed. 

The  Total  Module  Operability  for  thia  particular  EW 
Module  warn  computed  to  be  39.2  *.  Thia  computation  ia  as 
follows.  There  were  49  Subtaak  evaluated.  The  summation  of 
the  criticalities  of  these  Subtasks  in  order  to  accomplish 
the  mission  was  200.84.  By  assuming  a  perfect  layout,  we 
can  multiply  by  100  to  obtain  a  maximum  score  of  20,084. 
Next  the  Weighted  Operability  Values  were  summed  to  obtain 
the  actual  score  of  the  Module  of  7872.31.  When  the  actual 
score  is  divided  by  the  maximum,  an  indication  of  the 
effectiveness  of  the  layout  ia  obtained. 

Table  9  contains  an  ordering  of  the  Subtasks  by 
cumulative  weight.  This  was  determined  by  dividing  the 
Weighted  Deficit  Value  by  the  optimum  layout  effectiveness 
to  determine  how  much  the  deficit  each  Subtasks  comprises. 
These  were  then  ranked  from  most  to  least.  This  table  gives 
an  indication  of  which  Subtasks  should  be  improved  first  in 
order  to  achieve  the  most  cost  effective  approach  to 
improving  the  Module. 

Table  9  contains  the  rank  ordering  by  cumulative 
weights,  the  Subtask  number,  a  brief  description  of  the 
Subtask  and  its  associated  position,  the  operators  polled 
with  their  evaluation  of  the  Subtask  in  terms  of  Operator 
Workload  and  Space  Effectiveness  converted  to  an  interval 
scale,  the  operability  value  (the  mean  of  the  operators' 


TA3L2  9.  RANK  ORDER  OF  SUBTASKS  BY  CUMULATIVE  WEIGHT 
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73 

491 714 

38.2a 

4.143 

L71 

US 

2 

11.1 

tamtar  AutoCrritn  (17)  61  73  34  73  *  73 

24 

S.N8 

46.  IN 

4.2a 

L69 

6161 

2 

11.6 

Eval  S3  Brnq  (NTBS) 

78  74  74  a  a  43 

a 

57.143 

42.637 

4.2a 

LX 

8111 

34 

112 

Update  3  (NIBS) 

43  74  74  a  a  43 

a 

a.  714 

48.2N 

1371 

L41 

812 

2 

4.13 

Lop  Violations  (4JU 

a  a  34  46  78  34 

a 

49.2a 

38.714 

18N 

L23 

9177 

a 

1.13 

Update  3  (Sup) 

78  a  a  a  a  a 

51 

68.143 

39.637 

1714 

l  a 

9LX 

37 

111 

taport  a  (17) 

06  IN  96  96  a  96 

8 

66.429 

33.371 

4.2a 

1.18 

9116 

a 

112 

Provido  Oata  (17) 

a  a  74  74  a  78 

24 

67.4a 

22.371 

4.N8 

1.87 

94.0 

a 

112 

Estafi  EDI  tadns  (17) 

78  IN  IN  a  43  IN 

8 

71.714 

28.2a 

4.371 

LX 

93.0 

** 

4.1.3 

tamst  Eval  SI  (DUD 

34  34  34  IN  34  46 

a 

46.371 

31.429 

2.2a 

8.X 

910 

41 

11.2 

Estte  Op  toons  (17) 

a  IN  IN  IN  a  73 

a 

66.571 

19.714 

1637 

8.61 

97.X 

42 

1.1.1 

Assign  3JJ-17  (Sap) 

78  IN  IN  IN  a  a 

73 

aa.  ia 

19.837 

1167 

8.32 

97.28 

43 

113 

Assist  EDI  Enploy  (17) 

a  in  in  in  a  in 

a 

62.2a 

17.714 

1233 

141 

97.X 

44 

113 

tamtar  a  Entry  (17) 

78  IN  a  74  a  IN 

74 

78.371 

a.  429 

2.371 

8.43 

9121 

43 

111 

Engaqs  Trgts-SDI  (17) 

74  IN  a  74  IN  IN 

a 

68.637 

iLia 

4,714 

8.43 

9174 

46 

11.4 

tamtar  Cnvrfit  (17) 

a  in  in  in  a  in 

a 

aa.143 

11.637 

4.8N 

129 

9112 

47 

4.13 

Update  3  (MJD 

a  in  in  a  a  in 

74 

32.2a 

17.714 

2.429 

123 

9146 

46 

1.1.7 

tamtar  SLO-17  (Sup) 

a  IN  IN  IN  78  a 

— 

98.333 

9.667 

1667 

10 

9177 

49 

4.1.3 

DUE*  Isagss  (KJD 

78  IN  IN  IN  a  IN 

a 

17.714 

12.2a 

2.2a 

10 

1HN 

•valuations),  th«  deficit  value  <100  -  operability  value), 
the  aean  criticality,  the  cuaulative  weight  or  percentage  of 
the  total  deficit  that  that  particular  Subtask  comprises, 
and  the  total  percentage. 


* 


There  has  bssn  no  attempt  to  ascertain  what  Weighted 
Deficit  Value  or  Weighted  Operability  Value  ia  acceptable. 
This  is  beyond  the  scope  of  this  effort.  The  purpose  haa 
been  to  identify  which  areas  are  in  need  of  improvement  and 
what  areas  should  be  addressed  first  in  order  to  realize  the 
greatest  amount  of  improvement  for  a  given  effort.  To 
answer  the  question  of  what  Weighted  Deficit  or  Operability 
Value  ia  acceptable  will  call  for  additional  research 
targeting  the  Subtaska  individually  to  a  greater  detail  than 
was  attempted  here. 

A.  LINK  ANALYSIS  DISCUSSION 

The  Link  Analysis  results  show  that  there  is  only  one 
position  that  might  be  considered  acceptable  in  relation  to 
the  lengths  of  its  links.  This  ia  the  SLQ-17  position. 
This  can  be  seen  in  part  from  the  relatively  good  showing 
that  the  SLQ-17  console  position  had  in  comparison  to  the 
other  two  positions.  The  SLQ-17  operator  can  easily  view 
what  is  displayed  on  the  NTDS  console  and,  without  excessive 
movement,  view  the  WLR-1  displays.  He  is  within  good 
viewing  distance  of  the  NTDS  Status  Board  and  his  own  SLQ-17 
computer.  The  viewing  distance  to  the  WLR-1  position  and 
its  associated  Status  Board  are  rather  long,  but  still 
viewable.  Because  of  its  relatively  good  positioning  in 


relation  to  tha  raat  of  tha  Module,  SLQ-17  antriaa  ware  much 
lower  in  Table  9.  Thia  would  indicate  that  the  layout 
actually  proaotea  increaaed  operator  compensation  aince  the 
other  poaitiona  did  not  score  as  well.  A  score  of  39. 2*  la 
an  indication  of  a  poor  the  Module  layout  contributing  to  an 
Increaaed  operator  coapenaation  burden.  Were  the  module 
layout  better,  the  operators  would  have  felt  much  better 
about  the  Module  and  the  acore  would  have  been  higher. 

B.  OPERABILITY  ANALYSIS  DISCUSSION 

Several  observations  can  be  made  from  Table  9.  First, 
the  SLQ-17  appears  to  be  the  beat  position  in  the  EW  Module 
aince  its  first  entry  is  in  twenty-firat  place  in  the  table 
and  most  of  the  entries  are  at  the  bottom  of  the  table. 
Almost  27*  of  the  possible  improvements  can  be  made  in  the 
first  seven  entries  and  these  are  just  for  the  EW  Supervisor 
and  the  WLR-1  operator.  Note  that  the  criticalities  of 
these  Subtasks  ere  the  highest.  In  other  words,  these 
Subtaaks  which  are  very  critical  are  poorly  supported  by  the 
layout,  relative  to  the  less  critical  Subtasks.  Most  of  the 
lower  criticalities  are  associated  with  Subtasks  that  have  a 
relatively  good  layout  effectiveness. 

It  can  be  reasonably  argued  that  Module  Operability  of 
39. 2*  is  not  sufficient  for  an  EW  Module.  What  can  not  be 
argued  is  how  much  improvement  is  enough.  Nor  can  it  be 
extrapolated  from  this  study  what  improvement  a 
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it  can  ba  aaan  that 


raarrangaaant  can  raault  in.  However, 
iaprovaaant  can  ba  aada  in  cartain  areas,  aa  ia  indicatad  by 
caraful  paruaal  of  Tabla  9. 

C.  EXTRAPOLATION 

Further,  thia  approach  can  ba  uaed  for  poaaible 
axtrapolationa.  For  example,  comparing  Figura  1  and  Figura 
2,  similarities  ara  notad.  Thay  hava  tha  aama  arrangement 
of  poaitiona  <i.e.,  from  left  to  right.  WLR-1,  SLQ-17,  and 
NTDS).  The  poaitiona  ara  arranged  in  "straight  line'*  type 
of  layout.  Thia  raaultad  in  a  low  Layout  Eff activenesa 
rating  for  USS  CARL  VINSON.  It  may  ba  readily  conjectured 
that  another  arrangement  would  work  batter.  namely.  a 
"creacant"  ahapad  layout  with  tha  NTDS  in  between  WLR-1  and 
SLQ-1  and  tha  supervisor's  position  raised  and  directly  in 
back  of  tha  NTDS  operator. 

D .  CONCLUSION 

It  can  ba  concluded  that  tha  present  configuration  of 
tha  EW  Nodule  on  USS  CARL  VINSON  does  not  result  in  an 
optimal  utilization  of  thia  Nodule  in  terms  of  EW  mission 
accomplishment.  Further,  there  ia  a  real  need  to  assess  the 
layout  operability  of  the  warfare  modules  onboard  U.S.  Naval 
combatants.  This  thaais  has  provided  one  way  in  which  a 
measure  of  tha  af f ectivaneaa  of  a  particular  layout  can  be 
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determined.  Although  this  was  a  llaltad  test,  lndlcationa 
ara  that  thia  approach  works,  and  furthar  taating  is 
warrantad . 

Building  a  naw  layout  ia  urgad  with  tha  hops*  that  it 
aay  prova  by  taating  to  ba  battar  than  tha  laat  one,  uaing 
tha  Link  Analysis  and  Oparability  Analysis  illuatratad  in 
thia  work.  What  ia  significant  and  usaful  from  tha  Link 
Analysis  ia  that  any  improvamant  in  layout  design  should 
probably  start  with  ansuring  that  tha  critical  links  are  not 
ovarly  long  or  taxad  bayond  thair  limit.  Any  improvement  to 
the  layout  design  should  taka  into  account  these  critical 
links  to  reduce  them  to  their  optimum  and  any  changes  must 
not  adversely  affect  the  links  since  in  that  case  any  gain 
in  layout  design  nay  be  cancelled  by  a  loss  in  link 
utilization.  By  conducting  tests  at  landbased  test  sites, 
the  risks  of  error  are  reduced.  By  the  utilization  of  mock- 
ups  and  fleet  inputs,  the  risks  can  be  reduced  even  further. 
The  result  is  a  more  effective  layout  enhancing  mission 
accomplishment . 

E.  RECOMMENDATION 

It  is  recommended  that  a  land  based  test  facility  be 
established  that  would  incorporate  tha  ideas, 
recommendations,  and  methods  indicated  in  this  thesis  as  a 
first  step  to  upgrading  our  combat  workspaces. 
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The  purpose  of  this  questionnaire  is  a  subjective 
evaluation  of  the  layout  effectiveness  of  the  EW  Module  for 
use  in  an  algorithm  to  determine,  in  an  objective  sense,  the 
effectiveness  of  the  layout  in  accomplishing  the  mission  of 
the  Module.  To  do  this  there  is  a  series  of  subtasks 
differentiated  by  operator  that  must  be  assessed  in  terms  of 
operator  workload  per  aubtaak,  space  effectiveness  per 
subtask,  and  criticality  of  the  subtask  toward  overall 
mission  accomplishment.  What  is  required  is  to  make  this 
assessment  based  on  your  experience  and  expertise.  There  is 
no  time  limit,  you  may  ask  questions  of  anyone  you  wish,  and 
you  should  go  and  look  at  the  Module  to  make  sure  of  your 
answers  especially  if  you  are  unsure  of  some  of  the 
questions  concerning  movements.  There  are  no  right  or  wrong 
answers,  but  try  to  be  as  precise  as  you  can.  A  scenario, 
hopefully  similar  to  your  recent  operations  in  the  Sea  of 
Japan,  has  been  constructed.  For  each  of  the  aubtasks  on 
the  next  page,  mark  with  an  "X" ,  the  description  that  beat 
describes  the  operator  workload  (OW)  and  the  apace 
effectiveness  <SE) .  If  the  arrangement  of  the  space  has 


little  or  no  effect  on  aubtaak  accomplishment,  then  it  would 


rate  the  highest  <4) 


Conversely,  lx  the  layout  or 


arrangaaant  of  tha  apaca  negatively  iapacta  aubtaak 
accomplishment,  than  It  would  rata  a  (1>.  Give  your 
aaaaaaaant  of  tha  criticality  of  tha  aubtaak  in  ralation  to 
tha  ovarall  aiaaion  accoapliahaant.  Tha  daacriptiona  of 
criticality,  oparator  workload,  and  apaca  affactivanaaa  are 
liatad  on  a  aaparata  ahaat. 


SSSMABIfi. 

Thia  acanario  bagina  with  the  aaaumption  of  the  watch 
by  a  particular  section 

They  are  the  on-coming  watch  aection  in  the  EW  module 
of  a  NIMITZ  claaa  aircraft  carrier  that  ia  steaming  in  the 
open  ocean  with  aix  eacorta.  The  aacorta  are  one  VIRGINIA 
claaa  cruiser,  two  SPRUANCE  claaa  deatroyera,  one  OLIVER 
HAZARD  PERRY  claaa  frigate,  one  LOS  ANGELES  claaa  submarine, 
and  an  oiler.  There  are  heightened  tenaiona  world-wide  with 
a  probable  confrontation  between  tha  two  auper-powers. 
There  ia  a  Soviet  taak  group  within  200  NN.  The  task  group 
ia  comprised  of  a  KIEV  claaa  aircraft  carrier,  a  KIROV  claaa 
cruiser,  a  SOVERMENYY  class  destroyer,  two  KRIVAK  III  class 
frigatea,  a  SLAVA  class  destroyer,  and  three  auxiliaries. 
Additionally,  ECHO  II,  VICTOR  III,  and  OSCAR  class 
submarines  are  known  to  be  in  the  area  but  unlocated  for  the 
paat  twelve  hours.  A  Mod-KASHIN  ia  the  tattletale  for  the 


Battle  Group 


Both  forcas  art  within  rang*  of  Soviat  air 


powar . 

Ganaral  Quartara  ia  not  aat,  but  a  haightanad  Condition 
III  ataaming  watch  ia  aannad.  Thara  haa  baan  a  aoaantary 
lapaa  of  400  Hx  powar  and  tha  NTDS  ia  baing  raloadad.  Tha 
SLQ-17  naada  to  ba  raloadad  and  raprogramaad .  As  tha  NTDS 
ia  brought  on  tha  line,  tha  WLR-1  oparator  ia  told  to 
rachack  tha  paat  antriaa  in  hia  log  and  verify  that  thay  are 
atill  activa.  Aftar  15  minutes,  the  WLR-1  operator  reports 
that  ha  haa  intercepted  several  new  signals.  One  is  an 
airborne  sapping  and  reconnaiaaance  radar.  One  appeared  to 
be  a  brief  intercept,  of  a  submarine  radar.  Another  is  an 
air  search  radar  and  the  last  is  a  missile  acquistion  radar. 

The  NTDS  air  trackers  report  jamming  on  both  long  range 
and  3-D  air  search  radars. 

The  SLQ-17  alarms  and  displays  hostile  missile  symbols 
from  both  the  suspected  direction  of  the  SOviet  task  group 
and  two  angles  30  degrees  either  side  of  the  task  group. 

Deck  Launched  Interceptors  are  airborne  within  one 
minute. 

The  EW  operator  at  the  NTDS  console  is  entering  ESM 
bearing  lines  and  attempting  to  identify  unknown  contacts. 
The  SLQ-17  operator  shifts  operation  of  the  ECM  portion  to 
automatic  as  the  TAO  frees  weapons.  The  WLR-1  operator  is 
attempting  to  search  the  known  hostile  missile  homing  radar 
ranges  to  facilitate  identification.  General  Quarters  is 
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sounded.  Ths  TAO  orders  EMCON  to  be  set  for  battle  and  the 
WLR-1  operator  selects  EMCON  D  on  MUTE.  A  quick  check  of 
ooth  the  NTDS  scope  and  that  of  the  SLQ-17  indicates  that 
the  nusber  and  direction  of  the  inbound  unknowns  do  not 
natch.  The  EW  watch  section  tries  to  match  the  emerging 
identification  from  the  WLR-1  and  SLQ-17  to  both  the  SLQ-17 
and  the  NTDS  presentations. 

If  you  have  trouble  envisioning  this  scenario.  recall 
the  Sea  of  Japan  operations  on  your  last  deployment  and 
consider  the  signal  environment  and  tactics  you  saw  then. 
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WORKLOAD/COMPENSATION/ INTERFERENCE  (Wt< 


ital  and  Physical 


(1) 

Workload  Extra* 
Compensation  Extra* 
Interference  Extra* 


(2) 

Workload  High 
Compensation  High 
Intarfsrsnce  High 


(3) 

Workload  Nodarata 
Compensation  Nodarata 
Intarfaranca  Rodarata 


(4) 

Workload  Lon 
Compensation  Lon 
Intarfaranca  Lon 


SPACE  EFFECTIVENESS 

(1) 

(2) 

(3) 

(4> 

Inadequate  Performance 

Adequate  Performance 

Layout  Enhances  Spe- 

Layout  Design 

Due  to  Layout 

Achievable;  Layout 

cific  Task  Accomplish- 

Integrates 

Sufficient  to  Spec¬ 
ific  Task 

sent 

Multiple  Tasks 

CRITICALITY :  How  important  is  it  that  the  operator/team  be 
able  to  perform  thia  taak  as  compared  to  the  other 
taaka  in  aucceaafully  completing  the  miaaion? 

Sc?alg  .Va^yieS 


<1>  Of  very  amall  importance.  Ability  to  perform  thia 
taak  aa  compared  to  other  taaka  in  thia  duty  ia  unimportant, 
or  almoat  unimportant,  in  order  to  aucceaafully  complete  the 
miaaion  of  the  Module. 


<2>  Of  amall  importance.  Thia  taak  within  thia  duty  ia 
leaa  important  than  moat  taaka  required  to  aucceaafully 
complete  the  miaaion  of  the  Module. 


<3>  Of  moderate  importance.  Thia  taak  within  thia  duty  is 
about  aa  important  aa  moat  tasks  required  to  aucceaafully 
complete  the  miaaion  of  the  Module. 


<4)  Of  aubatantlal  importance.  This  task  within  thia  duty 
ia  more  important  than  moat  tasks  required  to  successfully 
complete  the  miaaion  of  the  Module. 

(5)  Of  extreme  importance.  Thia  taak  within  thia  duty  ia 
extremely  important  in  order  to  successfully  complete  the 
mission  of  the  Module. 


Taak :  1.1  Direct  ESH  March 

SubttlKl 

1.1.1  Aaalgn  aaareh  parameter a  to  SLQ-17 

Operator  Workload:  _____  _ ____  _____  _ 

<1>  <2>  <3>  <4> 

Space  E££  ectlveneaa :  _  _  _ 

<1)  (2)  (3)  (4) 

Criticality:  _ 

1.1.2  Aaalgn  aaareh  paraaatara  to  WLR-1 

Operator  Workload:  _____  _____  _____  _ 

<1)  <2>  <3)  <4) 

Space  E££ectiveneaa: _  _____  _  _ 

<1)  (2)  (3)  C4> 

Criticality:  _ 

1.1.3  Aaalgn  ESN  aanaor  report  reaponaibilltiea  --  own  ahip 

Operator  Workload:  _  _  _____  _ 

<1>  (2)  (3)  <4> 

Space  E££ectiveneea: _  _  _  _ 

<1>  (2)  (3>  <4) 

Criticality:  _ 

1.1.4  Aaalgn  ESN  aanaor  report  reaponaibilltiea  --  force 

Operator  Workload:  _  _  _  _ 

<1>  (2)  <3>  <4> 

Space  E££ectiveneaa: _  _  _  _ 

<1>  (2)  (3)  (4) 


Criticality 


1.1. S  Initiate  aonual  ID  raquaat  -  ahip 


Oparator  Workload: 


<1> 

<2> 

<3> 

<4> 

Spaca  Effactivanaaa: 

<1> 

(2) 

(3) 

<4> 

Criticality:  _ 

1.1.6  Initiata  manual  ID 

4> 

a 

• 

9 

1  7 

u 

-  forca 

Oparator  Workload: 

<1> 

(2) 

(3) 

<4> 

Spaca  Effactivanaaa: 

<1> 

(2) 

(3) 

(4) 

Criticality: 

1.1.7  Monitor  automatic  corralationa/aaaociationa. 

Oparator  Workload: 

<1) 

<2) 

(3) 

<4> 

Spaca  Effactivanaaa: 

<1> 

(2) 

C  3) 

(4) 

Criticality: 

Taak:  1.2  Raport/Diaaaminata  EW  Information 
SMbt&ahi 

1.2.1  Raport  avaluatad  EW  information 

Oparator  Workload : 

<1> 

(2) 

(3) 

(4) 

Spaca  Effactivanaaa: 

(1) 

(2) 

(3) 

<  4  ) 

Criticality: 
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1.2.2  Provide  EW  recomaendationa 

Operator  Workload:  _  _  _____ 

<1>  (2)  (3> 

Space  Effect! vene— :  ______  _ 

(1)  <2>  (3) 

Criticality:  _ 

1.2.3  Update  atatue  board  near  NTOS  conaole 

Operator  Workload:  _____  _  _ 

<1>  (2)  (3) 

Space  Ef f ectlveneaa :  ______  _ 

<1>  (2)  (3) 

Criticality:  _ 

1.2.4  Brief /debrief  eabarked  Airwinga 

Operator  Workload:  ______  _  ___ 

<1>  (2)  (3) 

Space  Effectiveneaa: _  _  _ 

<1>  <2)  (3) 

Criticality:  _ 

1.2.5  Navigation  by  paaaive  EW 

Operator  Workload:  _  _  _ 

<1)  (2)  (3> 

Space  Effectiveneaa: _  _  _ 

<1>  (2)  <  3) 

Criticality:  _ 


Teak:  1.3  Counter  Hoatile  Environment 

gufe&aglUL 


1.3.1  Promulgate  ECM  employment  criteria 


Operator  Workload:  _____  _  _  _ 

(1)  (2)  (3)  <43 

Space  Effectiveness: _  _____  _  _ 

(1)  <23  (3)  (4) 

Criticality:  _ 


NTDS  Operator 

Taak:  2.1  Collect  and  enter  EW  data  into  NTDS 
Subtaeka 

2.1.1  Enter.  Manual  ID  information  into  NTDS 

Operator  Workload:  _  _  _  _ 

<1>  (2)  (3)  (4) 

Space  Effectiveness: _  _  _  _ 

<1>  <2>  (3)  < 

Criticality:  _ 

2.1.2  Enter  manual  ESM/NTDS  track  aaaociationa 

Operator  Workload:  _  _  _  _ 

<13  <23  (33  <■ 

Space  Effectiveness: _  _  _  _ 

<13  <23  <33  < 

Criticality:  _ 

2.1.3  Perform  triangulation  of  ESM  bearing . lines 

Operator  Workload:  _  _  _  _ 

<13  <23  <33  <43 

Space  Effectiveness: _  _  _  _ _ 

<13  <23  (33  (43 


Criticality 


Operator  Workload:  _  _  _ 

< 1 )  (2)  (3) 

Space  Eff ectlveneae :  _____  _ 

(1)  (2)  <3> 

Criticality:  _ 

2.1.5  Adviae  operatora  of  bearing  resolution 

Operator  Workload:  _  _  _ 

<1>  <2>  <3> 

Space  Effectiveneaa: _  _  _ 

<1>  (2)  (3) 

Criticality:  _ 

2.1.6  Evaluate  externally  reported  ESM  bearings 

Operator  Workload:  . _  _  _  _ 

<1>  (2)  (3) 

Space  Effectiveneaa:  _ 

<1>  (2)  ( 3> 

Criticality:  _ 

Task:  2.2  Report /Diaaeninate  EW  Information 
Subtaak 

2.2.1  Report  evaluated  EW  information 

Operator  Workload:  _  _  _  _ 

<1>  (2)  (3) 

Space  Effectiveneaa: _  _ 

<1>  (2)  <3> 

Criticality:  _ 

2.2.2  Update  atatua  board  near  console 

Operator  Workload:  _  _ 

<1>  ( 2 )  ( 3 ) 

Space  Effectiveneaa: _  _ 

<1>  (2)  (3) 


Criticality 


Task:  3.1  Conduct  ESM  Search 
Subtaak 

3.1.1  Monitor  automatic  corralationa/aaaociationa 


Oparator  Workload: 


<1)  C2>  <3)  <4> 


Spaca  Effactivanaaa: , 


<1)  (2)  <3>  <4> 


Criticality:  _ 

3.1.2  Eatabliah  oparating  nodes  of  SLQ-17  <ESM) 


Oparator  Workload: 


<1)  (2)  (3)  <4> 


Spaca  Effactivanaaa: . 


<1>  (2)  (3)  <4) 


Criticality:  _ 

3.1.3  Enter  detection  and  response  parameters  (ESM/ECM) 


Operator  Workload: 


<1>  (2)  C3>  <4> 


Spaca  Effectiveness:. 


<1>  C2)  (3)  <4) 


Criticality:  _ 

3.1.4  Monitor  environment  on  NTDS  console 


Operator  Workload: 


<1)  (2)  <3>  <4> 


Space  Effectiveness:. 


<1>  (2)  (3)  <4> 


Criticality 


3.1.5  Evaluate  displayed  data 


Operator  Workload: 


<1> 

(2) 

(3) 

(4) 

Space  Effectiveness: 

(1) 

(2) 

(3) 

(4) 

Criticality: 

Task:  3.2  ReDort/Disseminate  EW  Information 
Subtask 

3.2.1  Report  evaluated  EW  information 

Operator  Workload : 

<1> 

(2) 

(3) 

(4) 

Space  Effectiveness: 

<1) 

(2) 

(3) 

C  4  ) 

Criticality: 

3.2.2  Provide  ESM/ECM  data 

to 

NTDS 

Operator  Workload: 

<1) 

(2) 

(3) 

(4) 

Space  Effectiveness: 

<1> 

(2) 

(3) 

(4) 

Criticality : 

3.2.3  Monitor  entry  of  EW 

data 

into  NTDS 

Operator  Workload: 

<1> 

(2) 

(3) 

(4) 

Space  Effectiveness: 

(1) 

(2) 

(3) 

C4> 

Criticality : 
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3.2.4  Update  statu*  board 


Oparator  Workload:  _  _  _  _ 

(1)  (2)  <3>  (4) 

Spaca  Effectiveness: _  _  _ 

(1)  <2>  (3>  (4> 

Criticality:  _ 


Taak:  3.3  Countar  Hoatila  Environment 

3.3.1  Engage  targets  with  ECM 

Operator  Workload:  _  _  _  _ 

<1>  (2)  <  3)  (4 ) 

Space  Effect  iveneaa: _  _  _  _ 

<1>  <2>  (3)  (4) 

Criticality:  _ 

3.3.2  Establish  ECM  operating  Modes 

Operator  Workload:  _  _  _  _ 

<1>  (2)  (3)  (4) 

Space  Effectiveness: _  _  _  _ 

<1>  <2>  (3)  (4) 

Criticality:  _ 

3.3.3  Assist  in  promulgation  of  ECM  employment  criteria 

Operator  Workload:  _  _  _  _ 

<1>  (2)  <3>  <  4 ) 

Space  Effectiveness: _  _  _  _ 

<1)  (2)  <  3)  (4) 


Criticality 


Task:  4.1  Conduct  ESM  Search 
Subtasks 

4.1.1  Saarch  assigned  bands 


Operator  Workload: 


<1> 

(2) 

<  3) 

Space  Effectiveness: 

<1> 

(2) 

(3) 

Criticalitv: 

4.1.2  Analyze  intercepted 

signals 

Operator  Workload: 

(1> 

<2> 

<3> 

Space  Effectiveness: 

<1> 

(2) 

<3) 

Criticalitv: 

4.1.3  Check  intercepts  for 

inages/harmonics 

Operator  Workload: 

<1> 

<2> 

C  3) 

Space  Effectiveness: 

<1> 

(2) 

(3) 

Criticalitv: 

4.1.4  Accurately  DF  intercepted 

signals 

Operator  Workload: 

<1> 

(2) 

<  3) 

Space  Ef f ectiveness : _ 

<1>  <2>  <  3) 


Criticality 
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4.1.5  Assist,  in  svaluating  ECU 


Opsrator  Workload: 


<1) 

<2> 

<3> 

Spacs  Effectiveness: 

(1) 

<2) 

(3) 

Criticality: 

Task:  4.2  Reoort/Disseminate  EW 
Subtaaks 

4.2.1  Provide  ESM  data  to  NTDS 

Information 

Operator  Workload: 

<1> 

(2) 

<3) 

Space  Effectiveness: 

<1> 

(2) 

(  3  > 

Criticalitv: 

4.2.2  Report  evaluated  EW 

information 

Operator  Workload: 

<1> 

(2) 

(3) 

Space  Effectiveness: 

<1> 

<2> 

(3) 

Criticality: 

4.2.3  Update  status  board 

near 

position 

Operator  Workload: 

<1) 

( 2  > 

(3) 

Space  Effectiveness:. 

<1)  <2>  (3) 


Criticality 


4.2.4  Log  all  intarcapta 


Oparator  Workload:  _____  ___ 

<1>  <2> 

Spaca  Effaetivanaaa: _  _ 

(1)  (2) 

Criticality:  _ 

Taak:  4.3  EMCON 

Satejfeaai  ± 

4.3.1  Monitor  EMCON 

Oparator  Workload:  _  _ 

<1>  (2) 

Spaca  Effaetivanaaa : _ 

<1> 

Criticality:  _ 

4.3.2  Raport  violation*  of  EMCON 

Oparator  Workload:  _ 

<1)  (2) 

Spaca  Effaetivanaaa: _ 

<1> 

Criticality:  _ 

4.3.3  Log  violations  of  EMCON 

Oparator  Workload:  _ 

<1>  (2) 

Spaca  Effaetivanaaa: _  _ 

<1>  (2) 

Criticality:  _ 

4.3.4  Monitor  MUTE 

Oparator  Workload :  _  _ 

(1)  (2) 

Spaca  Ef factivanaaa : _  _ 

<1>  (2) 


Criticality 


EW  Statu*  Board 

Task;  S.l  Maintain  Statua  Boarda 
Subtaaka 

5.1.1  Coaaunicata  with  oparatora 

Oparator  Workload :  _____  _____  _  _ 

(1)  (2)  (3)  (4) 

Spaca  Effactivanaaa: _  _____  _____  _ 

(1)  (2)  <3>  <4> 

Criticality:  _ 

5.1.2  Adviaa  oparatora  of  any  information  received 

Oparator  Workload:  _  _  _  _ 

<1)  (2)  (3)  <4> 

Space  Effectiveneaa: _  _  _  _ 

<1>  (2)  (3)  (4) 

Criticality:  _ 

5.1.3  Update  atatua  boarda 

Oparator  Workload:  _____  _____  _____  _ 

(1)  (2)  (3)  (4) 

Spaca  Effectiveneaa:  _  _  _ 

Cl>  <2>  (3)  (4) 

Criticality:  _ 
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For  ths  following  matrix,  rank  ths  blocks  from  best  to 
worst  <1  to  16).  The  lowest  numbered  block  is  the 
intersection  of  the  best  of  the  rows  and  columns.  The  number 


two  block  is  next  best,  and  so  on.  Note  the  arrows  and  the 
phrases  associated  with  them.  Design  means  the  design  of  the 
layout  or  arrangement.  Do  not  think  of  the  layout  of  one 
workstation,  such  as  the  WLR-*1  or  SLQ-17,  but  of  the  entire 
EW  Module.  Think  of  the  scenario  already  presented  in  order 
to  properly  consider  the  workload.  Ask  any  questions  you 
want  or  talk  among  yourselves  or  go  and  look  at  the  layout. 


Nultiple  Tasks 
Integrated 

Layout  Enhances 
Specific  Task 
Accoaolishment 


Performance 
Achievable 
(Layout  Sufficient) 

Inadequate 
Performance  due  to 
Layout,  Cannot  Ct-.- 
pensate  For  Sub- 
Par  Perfomance 


Workload  At  Workloao  Con-  Workload  Siigntly  Workload  As 

Critical  Level;  siderabiy  Higher  Higher  Than  Anticioated; 

Compensation  Than  Anticipated;  Anticioated;  No  Interference 

Very  Excessive  Cinper. sa:i:n  Moderate  No  Condensation 

High;  Interference  Compensation; 

Na.ior  Tircr  Interference 


WLR-1  Operator: 

How  many  times  in  an  eight  hour  watch  do  you: 

1.  Talk/communicate  with  the  SLQ-17  operator? 

2.  View  the  presentation  on  the  SLQ-17  console? 

3.  Talk/communicate  with  the  NTDS  operator? 

4.  View  the  presentation  on  the  NTDS  console? 

5.  View  the  NTDS  Status  Board  (SB>? 

6.  View  the  WLR-1  Status  Board? 

7.  Update  the  WLR-1  Status  Board? 

8.  Check  (visually)  the  SLQ-17  computer? 

9.  Reboot,  reset,  or  work  with  the  SLQ-17  computer? 

10.  Check  MUTE? 

11.  Change  any  settings  on  MUTE? 


SLQ-17  Operator: 

How  many  times  in  an  aight  hour  watch  do  you: 

1.  Talk/communicate  with  tha  NTDS  oparator? 

2.  Viaw  tha  praaantation  on  tha  NTDS  console? 

3.  Talk/comnunicate  with  tha  WLR-1  oparator? 

4.  Viaw  tha  praaantation  on  tha  WLR-1  console? 

5.  Viaw  tha  WLR-1  Status  Board? 

6.  View  tha  NTDS  Status  Board? 

7.  Update  tha  NTDS  Status  Board? 

8.  Check  (visually)  the  SLQ-17  computer? 

9.  Reboot,  reset,  or  work  with  the  SLQ-17  computer 

10.  Check  MUTE? 

11.  Change  any  settings  on  MUTE? 


NTDS  Operator/EW  Supervisor: 

How  many  times  in  an  eight  hour  watch  do  you: 

1.  Talk/communicate  with  the  SLQ-17  operator? 

2.  View  the  presentation  on  the  SLQ-17  console? 

3.  Talk/communicate  with  the  WLR-1  operator? 

4.  View  the  presentation  on  the  WLR-1  console? 

5.  Vic.  the  WLR-1  Status  Board? 

6.  View  the  NTDS  Status  Board? 

7.  Update  the  NTDS  Status  Board? 

8.  Check  (visually)  the  SLQ-17  computer? 

9.  Reboot,  reset,  or  work  with  the  SLQ-17  computer 

10.  Check  MUTE? 

11.  Change  any  settings  on  MUTE? 

12.  Communicate  outside  the  Module? 


Multiple  Tales 
Integrated 


73 


54 


Layout  Enhances 
Specific  Task 
Accomplishment 

40 

61 

74 

Adequate 

Perforeance 

Achievable 
(Layout  Sufficient) 

24 

45 

56 

Inadequate 

Perfonunce  due  to 
Layout,  Cannot  Com¬ 
pensate  For  Sub- 
Par  Perforeance 

0 

21 

34 

Workload  At 
Critical  Level; 
Compensation 

Vary  Excessive 

Workload  Con¬ 
sider  aoly  Higher 
Than  Anticipated; 
Compensation 

Higfij  Interference 
Major 

Workload  Slightly 
Higher  Than 
Anticipated; 
Moderate 
Compensation; 

Minor  Interference 

m 


as 


78 


46 


Workload  fis 
Anticipated; 

No  Interference 
No  Compensation 


NOTES:  The  following  results  are  the  Operability  Value  of 

ell  reeponeee  for  each  aubtaek.  Generally,  n  equalled  7  for 


the  aubtaaka,  although  there  were  some  with  only  aix 
reaponaea.  The  Standard  Deviation  ia  that  of  the  aaaple  and 
not  the  population  (i.e.,  the  standard  deviation  was 
calculated  using  n  vice  n-1) .  The  Interval  Scale  shown  was 
based  on  the  Ranking  Scale  which  the  EW  operators  provided 
through  the  questionnaire.  The  Ranking  (an  ordinal)  Scale 
was  then  converted  to  an  interval  scale  by  means  of  the 
Delta  Method  (a  linear  expansion) .  The  Criticality  listed 
is  a  mean  for  the  aubtaak.  The  Weighted  Operability  Value 
is  simply  the  Operability  Value  weighted  (multiplied)  by  the 
Criticality . 

EW  Supervisor 


Task :  1.1  Direct  ESM  search 
Subtaaka 

1.1.1  Assign  search  parameters  to  SLQ-17 

Operability  Value:  80.143 

Criticality:  3.167 

Weighted  Operability  Value:  253.81 

Weighted  Deficit  Value:  62.89 

1.1.2  Assign  search  parameters  to  WLR-1 

Operability  Value:  49.714 

Criticality:  4.143 

Weighted  Operability  Value:  205.97 

Weighted  Deficit  Value:  208.33 


1.1.3  Assign  ESM  ssnsor  report  responsibilities 


own  ship 


Operability  Value:  44.000 

Criticality:  4.300 

Weighted  Operability  Value:  198.00 

Weighted  Deficit  Value:  232.00 

1.1.4  Assign  ESM  sensor  report  responsibilities 


Operability  Value:  24.429 

Criticality:  4.000 

Weighted  Operability  Value:  97.72 

Weighted  Deficit  Value:  302.28 

1.1.3  Initiate  manual  ID  request  -  ship 

Operability  Value:  40.800 

Criticality:  4.250 

Weighted  Operability  Value:  173.40 

Weighted  Deficit  Value:  251.80 


1.1.8  Initiate  manual  ID  request  -  force 


Operability  Value: 
Criticality: 

Weighted  Operability  Value: 
Weighted  Deficit  Value: 


22.000 

4.000 

88.00 

312.00 


1.1.7  Monitor  automatic  correlationa/associations. 


Operability  Value:  90.333 

Criticality:  3.667 

Weighted  Operability  Value:  331.25 

Weighted  Deficit  Value:  35.45 

Task :  1.2  Report/Disseminate  EW  Information 
Subtaaka 

1.2.1  Report  evaluated  EW  information 


Operability  Value:  3.000 

Criticality:  5.000 

Weighted  Operability  Value:  15.00 

Weighted  Deficit  Value:  485.00 

1.2.2  Provide  EW  recommendations 

Operability  Value:  3.000 

Criticality:  5.000 

Weighted  Operability  Value:  15.000 

Weighted  Deficit  Value:  485.00 


-  force 


(SLQ-17) 


1.2.3  Update  atat.ua  board  naar  NTDS  conaola 


Oparability  Valua:  60.143 

Criticality:  3.714 

Walghtad  Oparability  Valua:  223.37 

Walghtad  Deficit  Valua:  146.03 

1.2.4  Brief /debrief  aabarkad  Alrwlnga 

Oparability  Valua:  29.286 

Criticality:  3.286 

Walghtad  Oparability  Valua:  96.23 

Walghtad  Oaficit  Value:  232.37 

1.2.3  Navigation  by  paaaiva  EW 

Oparability  Value:  17.500 

Criticality:  3.500 

Weighted  Operability  Value:  61.25 

Weighted  Deficit  Valua:  288.75 


Task.:  1.3  Counter  Hoatile  Environment 
Subtaaka 

1.3.1  Promulgation  of  ECU  employment  criteria 


Operability  Value:  52.500 

Criticality:  4.500 

Weighted  Operability  Value:  236.25 

Weighted  Deficit  Value:  213.75 


NTDS  Operator 

Taak:  2.1  Collect  and  enter  EW  data  into  NTDS 


Subtaaka 

2.1.1  Enter  manual  ID  information  into  NTDS 

Operability  Value:  9.857 

Criticality:  4.500 

Weighted  Operability  Value:  45.06 

Weighted  Deficit  Value:  405.64 

2.1.2  Enter  manual  ESM/NTDS  track  associations 

Operability  Value:  20.286 

Criticality:  4.714 

Weighted  Operability  Value:  95.63 

Weighted  Deficit  Value:  375.77 
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2.1.3  Perform  triangulation  of  ESH  bearing  line 


Operability  Value:  2S.000 

Criticality:  4.423 

Weighted  Operability  Value:  110.73 

Weighted  Deficit  Value:  332.18 

2.1.4  Enter  EW  flxea 

Operability  Value:  22.143 

Criticality:  5.000 

Weighted  Operability  Value:  110.72 

Weighted  Deficit  Value:  389.29 

2.1.5  Advise  operators  of  bearing  resolution 

Operability  Value:  23.714 

Criticality:  4.429 

Weighted  Operability  Value:  105.03 

Weighted  Deficit  Value:  337.87 

2.1.8  Evaluate  externally  reported  ESH  bearings 

Operability  Value:  57.143 

Criticality:  4.288 

Weighted  Operability  Value:  244.91 

Weighted  Deficit  Value:  183.69 


Tank:  2.2  Report/Disseminate  EW  Information 
2.2.1  Report  evaluated  EW  information 


Operability  Value:  34.714 

Criticality:  4.571 

Weighted  Operability  Value:  158.68 

Weighted  Deficit  Value:  298.42 

2.2.2  Update  status  board  near  console 

Operability  Value:  51.714 

Criticality:  3.571 

Weighted  Operability  Value:  184.67 

Weighted  Deficit  Value:  172.43 


SLQ-17  Operator 


Task :  3.1  Conduct  ESM  Search 
Subtask 


3.1.1  Monitor  automatic  corralatlona/aaaoclationa 


Oparablllty  Valua:  52.000 

Criticality:  4.286 

Walghtad  Oparablllty  Valua:  222.87 

Weighted  Deficit  Value:  205.73 

3.1.2  Eatabliah  operating  aodea  of  SLQ-17  (ESM> 

Oparablllty  Value:  80.571 

Criticality:  3.857 

Weighted  Operability  Valua:  310.76 

Weighted  Deficit  Value:  74.94 

3.1.3  Enter  detection  and  reaponaa  parameters  (ESM/ECM) 

Operability  Value:  45.286 

Criticality:  4.429 

Weighted  Operability  Valua:  200.57 

Weighted  Deficit  Value:  242.33 


3.1.4  Monitor  environment  on  NTDS  console 


Operability  Value:  88.143 

Criticality:  4.000 

Weighted  Operability  Value:  352.57 

Weighted  Deficit  Value:  47.43 

3.1.5  Evaluate  displayed  data 

Operability  Value:  46.000 

Criticality:  4.571 

Weighted  Operability  Value:  210.27 

Weighted  Deficit  Valua:  246.83 


Task:  3.2  Report/Diaaeminata  EW  Information 
3.2.1  Report  evaluated  EW  information 


Operability  Value:  66.429 

Criticality:  4.286 

Weighted  Operability  Value:  284.71 

Weighted  Deficit  Value:  143.89 

3.2.2  Provide  ESM/ECM  data  to  NTDS 

Operability  Value:  67.429 

Criticality:  4.000 

Weighted  Operability  Value:  269.72 

Weighted  Deficit  Value:  130.28 


3.2.3  Monitor  entry  of  EW  data  into  MTDS 


Operability  Value:  78.371 

Criticality:  2.371 

Weighted  Operability  Value:  202.00 

Weighted  Deficit  Value:  33.09 

3.2.4  Update  atatua  board 

Operability  Value:  24.837 

Criticality:  3.571 

Weighted  Operability  Value:  88.76 

Weighted  Deficit  Value:  268.34 


Taak:  3.3  Counter  Hoatile  Environment 
Subtask 

3.3.1  Engage  targets  with  ECM 

Operability  Value:  88.857 

Criticality:  4.714 

Weighted  Operability  Value:  418.87 

Weighted  Deficit  Value:  52.53 

3.3.2  Eatabliah  ECM  operating  Modes 


Operability  Value:  71.714 

Criticality:  4.571 

Weighted  Operability  Value:  327.80 

Weighted  Deficit  Value:  129.30 

3.3.3  Assist  in  proaulgation  of  ECM  esploysent  criteria 

Operability  Value:  82.286 

Criticality:  3.333 

Weighted  Operability  Value:  274.26 

Weighted  Deficit  Value:  59.04 


Task:  4.1  Conduct  ESM  Search 

Sufe&j&sjsa 

4.1.1  Search  assigned  bands 

Operability  Value:  3.000 

Criticality:  5.000 

Weighted  Operability  Value:  15.00 

Weighted  Deficit  Value:  485.00 


4.1.2  Analyze  intercepted  signals 


Operability  Value:  14.286 

Criticality:  5.000 

Weighted  Operability  Value:  71.43 

Weighted  Deficit  Value:  428.57 

4.1.3  Check  intercepts  for  laagea/haraonica 

Operability  Value:  87.714 

Criticality:  2.286 

Weighted  Operability  Value:  200.51 

Weighted  Deficit  Value:  28.09 

4.1.4  Accurately  DF  intercepted  signals 

Operability  Value:  11.286 

Criticality:  5.000 

Weighted  Operability  Value:  56.43 

Weighted  Deficit  Value:  443.57 

4.1.5  Aaslst  in  evaluating  ECM 

Operability  Value:  48.571 

Criticality:  2.286 

Weighted  Operability  Value:  111.03 

Weighted  Deficit  Value:  117.57 


Task:  4.2  Report/Dlsaeainate  EW  Inforaation 

4.2.1  Provide  ESN  data  to  NTDS 


Operability  Value:  3.000 

Criticality:  4.714 

Weighted  Operability  Value:  14.14 

Weighted  Deficit  Value:  457.26 

4.2.2  Report  evaluated  EW  inforaation 

Operability  Value:  7.857 

Criticality:  5.000 

Weighted  Operability  Value:  39.29 

Weighted  Deficit  Value:  460.72 

4.2.3  Update  atatua  board  near  poaltlon 

Operability  Value:  82.286 

Criticality:  2.429 

Weighted  Operability  Value:  199.87 

Weighted  Deficit  Value:  43.03 


4.2.4  Log  all  lntarcapta 


Oparabillty  Valua: 
Criticality: 

Walghtad  Oparabillty  Valua: 
Walghtad  Daficit  Valua: 


Taak:  4.3  EMCON 

4.3.1  Monitor  EMCON 

Oparabillty  Valua: 
Criticality: 

Walghtad  Oparabillty  Valua: 
Walghtad  Daficit  Valua: 

4.3.2  Raport  violationa  of  EMCON 

Oparabillty  Valua: 
Criticality: 

Walghtad  Oparabillty  Valua: 
Walghtad  Daficit  Valua: 

4.3.3  Log  violationa  of  EMCON 

Oparabillty  Valua: 
Criticality: 

Walghtad  Oparabillty  Valua: 
Walghtad  Daficit  Valua: 

4.3.4  Monitor  MUTE 

Oparabillty  Valua: 
Criticality: 

Walghtad  Oparabillty  Valua: 
Walghtad  Daficit  Valua: 


Taak :  S.l  Maintain  Status  Boardi 
§M 

5.1.1  Coaaunicata  with  oparatori 

Oparabillty  Valua: 
Criticality: 

Walghtad  Oparabillty  Valua: 
Walghtad  Daficit  Valua: 


13.429 

4.571 

81.38 

395.72 


46 . 429 
4.286 
198.99 
229.61 


44.143 

3.857 

170.26 

215.44 


49.286 

3.000 

147.86 

152.14 


35.571 
3.714 
132.11 
239 . 29 


10.573 
4.571 
48.33 
408 . 77 


5.1.2  Adviaa  operators  of  any  information  racaivad 


Oparability  Valua: 
Criticality: 

Weighted  Oparability  Valua: 
Weighted  Deficit  Valua: 

5.1.3  Update  atatua  boar da 

Oparability  Valua: 
Criticality: 

Waightad  Oparability  Valua: 
Waightad  Deficit  Value:: 


24.429 

4.000 

97.72 

302.28 


20.289 

4.714 

95.64 

375.76 


This  appendix  gives  a  brief  description  of  the  delta 
method,  an  algorithm  for  converting  ordinal  measures  on  the 
cells  of  a  matrix  to  a  scale  with  interval  properties 
satisfying  the  conditions  of  additive  conjoint  measurement. 
This  appendix  is  essentially  the  same  as  that  in  Ref.  2, 
only  being  changed  to  reflect  the  present  application.  For 
a  further  and  fuller  description,  see  Coombs  CRef .  6] .  The 
method  will  be  briefly  described  using  the  matrix  in  Figure 
C-l.  This  matrix  is  similar  in  form  to  the  4x4  matrix  used 
in  the  OW/SE  rating  matrix  developed  for  the  Subtask 
Questionnaire  but  smaller  in  size. 


Factor  II  Q 


Factor  I 


Figure  C-l 


In  Figure  C-l,  Factor*  I  and  II  rapraaant  two 
indapandant  aaaau ras,  and  tha  nuabar*  in  tha  calls  of  tha 
matrix  rapraaant  an  ampirical  ordering  of  overall 
performance  over  combinations  of  factors  I  and  II.  Higher 
numbers  represent  batter  overall  performance. 

The  resulting  scales  will  be  interval  measures  of  I  and 
II  as  well  as  overall  performance.  Because  the  scale  is 
additive,  the  measure  of  overall  parformance  of  any  cell 
must  be  the  sum  of  the  corresponding  row  and  column  scale 
values.  Furthermore,  the  resulting  performance  measure  must 
reflect  the  ordering  of  the  cells  of  the  matrix. 
Consequently,  any  set  of  scale  values  which  provide  an 
additive  representation  for  a  matrix  must  simultaneously 
satisfy  the  equations  implied  by  the  additive  representation 
and  the  inequalities  specified  by  the  rank  ordering  of  the 
cells  of  the  matrix.  Conditions  under  which  a  set  of  linear 
equations  and  inequalities  have  a  common  solution  are 
specified  mathematically  by  the  Theorem  of  the  Alternative. 
In  practice,  solutions  may  be  found  by  using  various  linear 
programming  techniques.  The  delta  method  is  one  such 
technique  that  is  simple  enough  to  be  done  by  hand  for  small 
matrices. 

The  delta  method  proceeds,  in  general,  as  follows. 
Cells  in  the  matrix  are  initially  given  arbitrary  positive 
scale  values  (represented  by  the  Greek  letter,  delta;  hence, 
the  name) (we  will  replace  delta  with  the  Roman  letter  d,  for 
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of  computation)  which  satisfy  ths  aquations  apacifisd 


by  tha  additiva  rapraaantation .  For  example,  if  B  wars 
assigned  tha  valua  di  and  Q  wara  aaaignad  tha  valua  d3 ,  than 
tha  call  BQ  would  hava  tha  valua  di  ♦  dg.  Aftar  initial 
aaaignaanta  ara  nada,  tha  ralationahipa  batwaan  scala  values 
and  tha  d'a  ara  changad  to  taka  into  account  the  constraints 
given  by  tha  rank  ordaring  of  tha  matrix.  Whan  the 
procedure  ia  completed  and  a  solution  is  found,  scale  values 
ara  represented  by  positive  linear  combinations  of  the  d's, 
such  that  any  choice  of  positive  d's  will  lead  to  scale 
values  which  satisfy  both  tha  aquations  implied  by 
additivity  and  tha  inequalities  implied  by  rank  order. 

The  levels  of  each  factor  ara  assigned  values  which 
reflect  the  ordering  on  that  factor.  Thus  for  Factor  I,  A 
will  be  assigned  a  value  of  0  since  it  is  the  lowest  level 
of  tha  factor.  B  will  be  assigned  an  arbitrary  positive 
value  of  di.  Since  C  is  a  higher  level  of  performance  than 
B,  it  may  be  expressed  as  di  ♦  d2  where  d2  ia  an  arbitrary 
positive  constant  representing  the  difference  between  the 
scale  values  of  B  and  C.  Similarly,  in  Factor  II,  P,  Q,  and 
R  nay  be  assigned  values  of  0,  dg,  and  d4,  respectively. 
Since  the  overall  performance  is  an  additive  combination  of 
the  two  factors,  the  scale  value  of  individual  cells  may  be 
stated  in  terms  of  the  d's  as  has  been  done  in  Figure  C-2. 


i  rut 


d3+d4  R 


d3«’d4 


di»d3^d4 


di*d24,d3+d4 


1 

1 

i 

1 

1 

1 

1 

Factor 

II 

7" 

d3  0  1 

I 

1 

d3 

1 

1  di-*-d3 

1 

l 

1 

1  di»d2+d3 

1 

1 

1 

1 

1 

1 

1 

0  P  1 

1 

1 

0 

1 

1  di 

1 

l 

1 

1  di*d2 

1 

1 

1 

1 

1 

l 

ABC 
0  di  di*d2 

Factor  I 
Figure  C-2 

Alternately,  the  scale  values  say  be  displayed  as  in 
Figure  C-3.  this  figure  represents  the  work  sheet  which 
will  be  used  in  the  algorithm.  On  the  top  half  of  the 
figure  are  the  individual  factor  scale  values.  On  the 
bottom  half  are  the  values  of  overall  performance,  listed  in 
decreasing  rank  order.  The  columns  represent  the  four  d's. 
the  numbers  in  the  work  sheet  are  the  coefficients  in  the 
equation : 

Scale  Value  *  coefi  x  dj  ♦  coef 2  x  d2  ♦  coef3  x  d3  ♦  coef4  x 
d4 . 

A  blank  indicates  a  coefficient  of  zero. 
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The  general  procedure  Involves  compering  cells  adjacent 
in  the  rank  ordering  of  the  matrix  and  redefining  the 
relationship  between  the  scale  values  and  the  d's  so  that 
the  order  of  the  cells  will  be  preserved  for  any  choice  of 
positive  d's.  the  cells  may  be  examined  in  any  order;  in 
this  example  and  the  Layout  Ef f ecti veness  application,  we 
will  start  from  the  lowest  performance  and  proceed  to  the 
highest  level.  This  involves  moving  from  the  bottom  to  the 
top  of  the  work  sheet. 


The 

first  pair  of  cells 

in  BP  and  AP. 

examination 

of 

Figure 

C-3  shows  that  BP  is 

higher  than  AP. 

Since  BP 

has 

value  di  and  AP  haa  value  0,  it  aay  be  concluded  that  di  > 


0.  Thia  inequality  ia  clearly  aatiafied  for  any  positive 
value  for  dj.,  ao  it  ia  not  neceaaary  to  redefine  thia  value. 
It  will  be  recalled  that  valuea  were  assigned  to  A  and  B  so 
that  B  would  have  a  higher  value  than  A. 

The  aecond  inequality  ia  AQ  >  BP.  Subatituting  the 
valuea  in  thia  inequality  givea  d3  >  dj,.  Thia  inequality  is 
not  true  for  all  valuea  of  d3  and  di.  However,  since  d3  > 
dj.,  it  possible  to  replace  d3  by  di  ♦  d3' ,  for  positive  d3' . 
Now,  for  any  choice  of  positive  di  and  d3',  the  inequali  ty 
di  ♦  d3 '  >  di  holds.  On  the  work  sheet,  d3  is  replaced  by 
di  ♦  d3# ;  that  ia,  in  any  row  with  a  d3,  a  di  and  a  d3'  are 
added  and  the  d3  ia  deleted.  For  convenience,  and  because 
the  d3'  column  looks  exactly  like  the  d3  column,  the  d3' 
column  is  put  where  d3  was.  Thia  is  merely  a  relabeling  of 
columns.  Note  that  as  many  'marks'  as  were  in  the  d3  column 
are  added  to  the  di  column.  The  work  sheet  at  this  point 
looks  like  Figure  C-4. 

The  next  inequality,  BO  >  AO  implies  2di  ♦  d3'  >  dj, 
d3'  or  di  >0.  Again  it  is  not  necessary  to  make  any 
changes  to  satisfy  thia  constant. 
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La  -‘-a  ^  ^  -• 


di  d?  da'  da 


A 

1 

i 

I 

f 

! 

i 

i 

B 

il _ 

i 

i 

1 

i 

i 

_ !_ 

C 

il _ 

il 

• 

i 

i 

i 

_ L 

P 

J _ 

i 

i 

i 

i 

i 

_ L 

0 

1 1 

i 

il 

i 

i 

i 

R 

i ; 

i 

J 1 

il 

i 

i 

i 

CR 

111  11 

11 

11 

1 

1 

1 

BR 

1  1 

111  1 

1 

>1 

1 

11 

1 

1 

1 

1 

1 

1 

CQ 

1  1 

111  11 

1 

1 1 

1 

1 

1 

1 

1 

1 

1 

1 

AR 

1  1 

11  1 

1 

i; 

1 

11 

1 

1 

1 

1 

1 

1 

CP 

1  1 

11  11 

i 

i 

1 

1 

1 

l 

1 

1 

1 

BQ 

1  1 

JJ.1  1 

i 

>1 

1 

1 

1 

l 

1 

1 

1 

1 

AQ 

1  1 

11  1 

i 

1 1 

1 

1 

1 

l 

1 

1 

1 

1 

AQ>BP»d3>di 

BP 

1  1 

11  l 

i 

i 

1 

1 

1 

l 

1 

1 

1 

1 

,d3*di+d3' 

BP>AP*di>0 

AP 

1  1 

l  l 

i 

i 

1 

1 

1 

I 

1 

1 

1 

1 

do  nothing 

Figure 

o 

i 

* 

Proceeding  up 

the 

work 

sheet . 

the 

next  inequality  is  CP 

This  implies  that  d2 

>  di 

♦  d3' 

a 

Here.  we  do  the 

same  as  before.  in  that  we  redefine  d2  «*  dl  ♦  d3'  ♦  d2' » 
and  consequently  replacing  d2  in  every  row  in  which  it 
occurs  by  a  di,  d3' .  and  d2'*  Again  d2'  is  put  in  the 
column  where  d2  was.  Consequently.  the  procedure  involves 
relabeling  the  d2  column  d2'  and  adding  a  di  and  a  d3'  for 
each  d2  in  any  row  in  which  a  d2  appears.  The  work  sheet  at 
this  point  looks  like  Figure  C-5. 
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The  following  throw  stop  nothod  is  usod  to  rodofino  tho 
d's.  First*  d4'  is  split  into  two  parts*  d^' '  and  d«j. 
Sines  di  ♦  d3'  >  d4' *  this  division  say  bo  arbitrarily  done 
so  that  di  >  d4' '  and  d3'  >  dg.  Tho  preceding  two 
inequalities  nay  bo  handled  by 


di  do*  da*  d a' 


A 

1 

1 

i 

i 

i 

i 

i 

B 

11 

1 

i 

i 

i 

i 

i 

C 

111 

11 

il 

i 

I 

i 

i 

P 

1 

1 

i 

i 

i 

i 

i 

Q 

11 

1 

il 

i 

i 

i 

i 

R 

Ilk- 

_L1 _ 

JJL _ 

1 1 

i 

i 

i 

CR 

Hill 

J-ll 

111 

1  1 

1 

1 

1 

BR 

1 

1111 

1 

1 

1  1 

1 

1  1 

1 

1 

1 

1 

1 

1 

CO 

1 

1  111 

1 

11 

1 

111 

1 

1 

1 

1 

1 

1 

1 

1 

AR 

1 

111 

1 

11 

1 

11 

1 

1  1 

1 

1 

1 

1 

1 

1 

AR>CP  =  d4>di ♦d2 ' 

CP 

I 

111 

1 

1  1 

1 

11 

1 

1 

1 

1 

1 

1 

1 

1 

d4=di«-d2'  *d4 ' 
CP>BQ=d2>di-d3' 

BQ 

1 

111 

1 

1 

1 

11 

1 

1 

1 

1 

1 

1 

1 

1 

d2=di *-d3' *d2 ' 
BQ>AQ=di>0 

AQ 

1 

1  1 

1 

1 

1 

1  1 

1 

1 

1 

1 

1 

1 

1 

1 

do  nothing. 
AQ>BP=d3>di 

BP 

1 

11 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

d3=di -d3 ' 
BP>  AP*d]_  >0 

AP 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

do  nothing. 

Figure  C-6 


previously  discussed  methods.  Thus  the  three  replacements 


Now  for  any  choica  of  poaitiva  d'a,  CQ  >  AR  la 


aatiafiad.  Tha  work  ahaat  now  looka  lika  Figura  C-7. 

Tha  aathoda  of  handling  tha  ramaining  inaqualitiaa  hava 
alraady  baan  diacuaaad.  Nota  that,  tha  atapa  to  coaplata  the 
workahaat  ia  (matrix  aiza  -  1),  or  in  tha  exampla.  8.  For 
tha  Modula  Oparability  application  thara  wara  15  atapa. 

Whan  coaplatad.  tha  top  half  of  tha  work  ahaat  shows  the 
ralationahip  batwaan  acala  valuaa  and  tha  newly  defined  d'a. 
For  tha  axampla  given,  tha  following  relationahipa  hold: 

A  *  0 


B  « 

dl'  ♦ 

d3" 

♦  d4'" 

C  « 

2di '  ♦ 

<*2' 

♦  3d3"  ♦ 

2d4"'  h 

ds 

P  « 

0 

Q  « 

di'  ♦ 

2d3'  ' 

♦  d4'"  ♦ 

<*5 

R  - 

2di'  ♦ 

<*2' 

♦  4d3"  ♦ 

3d4  *  *  * 

►  2ds 

di 

do* 

ds'  da' 

d* 

A 

l 

1 

l  1 

l  1 

1 

B 

11 

1 

1  l 

1  1 

C 

111 

n 

11  111 

11  1 

1 

P 

1 

i 

I  1 

1  l 

1 

0 

11 

i 

11  11 

n  i 

R 

.LLL 

LI— 

■  i  iin.. 

LU  .1 

- L 

CR 

1 1111 

ill 

ni  ni 

15 _ Lilli 

BR 

I 

1111 

i 

n 

i  i 

n  mil 

l  1 

111  1 

i 

1  (d4'«d4"*ds 

i 

i 

i  i 

1  l 

1  (di*d4' ' ♦di ' 

CQ 

mi 

n 

in  mi 

111  1 

1  C0>AR»di»d3'>d4' 

i 

i 

i  i 

1  1 

1  Cd3'*d5^d3'' 

AR 

ni 

n 

n  mi 

111  I 

1  AR>CP*da>di *d2' 

i 

i 

i  i 

l  1 

1  d4»di»d2' ♦d4 ' 

CP 

ni 

n 

n  in 

n  i 

1  CP>BQ*d2>di  *d-3' 

i 

i 

i  i 

i  i 

1  d2*di*d3' *d2' 

BQ 

in 

i 

n  in 

n  i 

1  BQ>AQ«di>0 

i 

i 

i  i 

i  i 

1  do  nothing. 

AQ 

n 

i 

n  n 

n  « 

1  AG>BP*d3>di 

i 

i 

i  i 

i  i 

1  d3»di*d3' 

BP 

n 

i 

i  n 

i  i 

1  BP>AP»di>0 

i 

i 

i  i 

i  i 

I  do  nothing. 

AP 

J _ 

J _ 

i  i 

i  i 

1 

Figura  C-7 


Any  choice  of  positive  d ' a  will  give  a  seal*  for  Factors  I 
and  II  as  wall  as  ovarall  parfornanca  which  is  an  additiva 


raprasantation,  and  in  which  tha  ovarall  ordarlng  agraas 
with  tha  aspirical  ordaring.  Figura  C-8  contains  tha 
rasults  of  this  example. 
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Figure  C-8 

A  common 

choice 

of  d' 

s  is  to  make 

them 

all 

equal 

to 

This  choica  for  d  yialds  a  sat  of  scala  values  which 
raprasant  tha  sat  of  Minimal  intagara  which  will  produce  the 
requires  rank  order  of  tha  matrix  calls. 

Tha  completed  work  sheet  for  tha  Module  Operability 
application  follows.  Note  that  tha  numbers  within  tha  d's 
are  expressed  in  arabic  numerals  for  viewing  ease  and  the 
totals  are  listed  on  tha  side. 
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